-- Logs begin at Mon 2022-06-06 13:37:40 UTC, end at Mon 2022-06-06 14:43:45 UTC. -- Jun 06 14:06:20.826698 dv-cinder-844140-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 06 14:06:20.836309 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 6 14:06:20 2022] *** Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: nodename: dv-cinder-844140-2-iscsi Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: machine: x86_64 Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: clock source: unix Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: pcre jit disabled Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: detected number of CPU cores: 4 Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: current working directory: / Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: detected binary path: /usr/bin/uwsgi-core Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: your processes number limit is 47680 Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: your memory page size is 4096 bytes Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: detected max file descriptor number: 2048 Jun 06 14:06:20.838469 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: lock engine: pthread robust mutexes Jun 06 14:06:20.839645 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: thunder lock: enabled Jun 06 14:06:20.839645 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 06 14:06:20.839645 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 06 14:06:20.884034 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: Python main interpreter initialized at 0x5648737939c0 Jun 06 14:06:20.884034 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: python threads support enabled Jun 06 14:06:20.884034 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: your server socket listen backlog is limited to 100 connections Jun 06 14:06:20.884034 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: your mercy for graceful operations on workers is 90 seconds Jun 06 14:06:20.884429 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: mapped 403077 bytes (393 KB) for 2 cores Jun 06 14:06:20.884504 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: *** Operational MODE: preforking *** Jun 06 14:06:20.884809 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: *** uWSGI is running in multiple interpreter mode *** Jun 06 14:06:20.884809 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: spawned uWSGI master process (pid: 80205) Jun 06 14:06:20.885587 dv-cinder-844140-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 06 14:06:20.887232 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: spawned uWSGI worker 1 (pid: 80214, cores: 1) Jun 06 14:06:20.887232 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: spawned uWSGI worker 2 (pid: 80215, cores: 1) Jun 06 14:06:20.887232 dv-cinder-844140-2-iscsi devstack@keystone.service[80205]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 06 14:06:26.694030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 14:06:26.694030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: from cryptography.utils import int_from_bytes Jun 06 14:06:26.695054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 14:06:26.695054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: from cryptography.utils import int_from_bytes Jun 06 14:06:26.715360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 14:06:26.715360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: from cryptography.utils import int_from_bytes Jun 06 14:06:26.716116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 14:06:26.716116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: from cryptography.utils import int_from_bytes Jun 06 14:06:26.746892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 06 14:06:26.747129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 06 14:06:26.747261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] command line args: [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 06 14:06:26.747512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 06 14:06:26.747694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ================================================================================ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 06 14:06:26.747952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] admin_token = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.748198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] config_dir = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.748381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.748564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] config_source = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.748764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] debug = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.748971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.749349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.749511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.749766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.750018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.750252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.750433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_config_append = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.750611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_dir = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.750965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_file = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.751206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.751463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.751689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.752073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.752306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.752306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.752790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.752790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.753239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.753239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.753563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.753833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.754095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.754095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.754491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.754871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.755061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] publish_errors = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.755333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.755613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.755836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.756042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.756190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.756465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] standard_threads = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.756655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.756809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.756940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.757084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_journal = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.757261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_json = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.757438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_stderr = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.757628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_syslog = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.757844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.758113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.758350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.758616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.758856 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.759105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.759348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.759600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.759899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.760684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.760942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.761193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.761441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.761732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.761990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.762245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.762494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.762737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.762994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.763249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.763506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.763742 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.764047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.764313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.764567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.764827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.765058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.765296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.765530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.765792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.766046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.766296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.766534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.766764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.767014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.767265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.767504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.767759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.767990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.768216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.768455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.768687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.768922 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.769154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.769285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 06 14:06:26.769399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 06 14:06:26.769498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.769595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] command line args: [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 06 14:06:26.769692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.769813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 06 14:06:26.769943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.770093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ================================================================================ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 06 14:06:26.770207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.770325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] admin_token = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.770440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.770560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] config_dir = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.770684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.770807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.770923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.771058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] config_source = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.771242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.771430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] debug = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.771430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.771801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.771967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.772147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.772325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.772600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.772600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.772957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.773089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.773253 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.773427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.773594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.773771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.773913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_config_append = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.774090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.774265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.774391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.774583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_dir = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.774754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.774897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_file = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.775042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.775159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.775273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.775418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.775589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.775768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.775965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.775965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.776326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.776494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.776640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.776760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.776899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.777035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.777195 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.777385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.777547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.777547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.777875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.778034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.778203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.778203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.778436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.778436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.778704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.778896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.779143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.779309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.779460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.779584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.779740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.779740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.779740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.780127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.780308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.780448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] publish_errors = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.780641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.780791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.780939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.781073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.781173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.781327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.781506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.781648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.781809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.781981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.782156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.782277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] standard_threads = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.782455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.782647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.782784 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.782940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.783042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.783140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.783272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.783434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_journal = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.783583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.783728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_json = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.783916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.784089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_stderr = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.784089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_syslog = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.784374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.784374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.784629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 14:06:26.784772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.784917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.785034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.785141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.785240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.785240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.785448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.785585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.785766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.785905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.786015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.786265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.786424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.786608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.786742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.786861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.787022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.787212 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.787212 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.external = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.787508 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.787699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.787837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.787964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.788088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.788229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.788422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.788591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.password = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.788752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.788947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.token = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.789116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.789281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.789281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.789538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.789656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.789793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.789928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.790045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.790161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.790288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.790450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.790635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.790762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.790900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.791062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.791062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.791322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.791495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.791667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.791845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.792025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.792169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.792315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.792461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.792610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.792749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.792895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.793050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.793050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.793365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.793518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.793675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.793863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.794005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.794137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.794280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.794464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.794624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.794765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.795012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.795138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.795294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.795472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.795652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.795831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.796021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.796205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.796387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.796570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.796747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.796747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.797098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.797272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.797457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.797641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.797844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.798029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.798250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.798454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.798654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.798842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.799026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.799026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.799375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.799547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.799670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.799836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.800018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.800158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.800276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.800432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.800621 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.800815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.800966 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.801609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.802794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.802978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.803136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.803277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.803415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.803625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.803808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.803976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.804120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.804239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.804368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.804497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.804631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.804772 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.804898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.805070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.805258 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.805446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.805446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.805740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.805740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.806048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.806048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.806397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.806397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.807190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.807353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.807513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.807674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.807853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.808035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.808223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.808395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.808534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.808683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.808844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.808974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.809105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.809276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.809441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.809619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.809915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.810060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.810233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.810371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.810371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.810647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.810799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.810929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.811098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.811245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.811379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.811524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.811704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.811891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.812046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.812046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.812344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.812474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.812610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.812771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.812963 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.813150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.813345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.813535 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.813813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.814006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.814185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.814331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.814482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.814644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.814795 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.814943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.815120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.815290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.815527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.815644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.815771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.815902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.815902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.816194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.816356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.816547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.816743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.816884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.817020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.817157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.817275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.817391 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.817518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.817660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.817849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.817849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.818109 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.818262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.818463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.818611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.818781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.external = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.818923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.819043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.819160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.819286 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.819406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.819406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.819623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.819767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.819966 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.password = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.820124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.820270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.token = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.820430 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.820601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.820765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.820961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.821153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.821298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.821432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.821605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.821838 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.821838 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.822183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.823023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.823227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.823411 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.823547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.823748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.823891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.824047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.824047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.824395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.824574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.824748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.824913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.825159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.825349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.825349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.825638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.825856 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.826039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.826227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.826414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.826589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.826769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.826964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.826964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.827316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.827469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.827604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.827787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.827964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.828112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.828257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.828382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.828561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.828739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.828739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.829077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.829268 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.829457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.829652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.829899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.830107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.830285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.830470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.832267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.832474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.832673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.832888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.833080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.833275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.833433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.833555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.833555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.833826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.833951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.834088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.834205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] role.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.834321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.834438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] role.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.834564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.834681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.834799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.834916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.835070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.835260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.835403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.835562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.835734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.835915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.835915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.836218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.836414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.836542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.836676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.836797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.836953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.837104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.837298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.837474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.837649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.837649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.837649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.838236 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.838378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.838517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.838648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.838795 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.838982 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.838982 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.839302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.839429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.839429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.839751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.839880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.840080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.840214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.840214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.840492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.840674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.840674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.840968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.841111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.841328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.841496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.841636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.841839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.842011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.842157 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.842293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.842415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.842591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.842797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.842797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.843061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.843228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.843551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.843719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.843719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.844023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.844171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.844171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.844444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.844604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.844780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.844949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.844949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.845174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.845344 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.845344 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.845631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.845833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.845833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.846164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.846300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.846479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.846666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.846926 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.847061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.847179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.847339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.847339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.847587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.847754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.847880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.848016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.848148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.848148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.848395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.848594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.848743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.848887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.849035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.849155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.849272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.849389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.849538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.849700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.849886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.850012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.850164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.850350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.850505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.850652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.850811 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.850935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.851091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.851215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.851342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.851459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.851576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.851692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.851861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.851861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.852142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.852300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.852453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.852611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.852789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.852937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.853080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.853213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.853382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.853575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.853575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.853963 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.854167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 06 14:06:26.854311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.854311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.854311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.854311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.854311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.854311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.854311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.854311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.854311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.855499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.855499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.855499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.855499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.855499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.855499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.855499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.855499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.855499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.857039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.857039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.857039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.857039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.857039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.857039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.857039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.857039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.857039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.858552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.858552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.858552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.858552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.858552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.858552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.858552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.858552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.858552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.860050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.860050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.860050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.860050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.860050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.860050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.860050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.860050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.860050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.861158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.861158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] role.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.861158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] role.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.861158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.861158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.861158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.861158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.861158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.861158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.862087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.862087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.862087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.862087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.862087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.862087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.862087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.862087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.862087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.863009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.863009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.863009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.863009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.863009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.863009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.863009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.863009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.863009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.864061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.864061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.864061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.864061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.864061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.864061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.864061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.864061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.864061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.865316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.865316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.865316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.865316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.865316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.865316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.865316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.865316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.865316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.866587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.866587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.866587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.866587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.866587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.866587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.866587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.866587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.866587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.867892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.867892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.867892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.867892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.867892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.867892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.867892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.867892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.867892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.869910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.869910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.869910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.869910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.870463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.870669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.870925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.871064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.871190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.871421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.871421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.871421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.871421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 14:06:26.871421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 06 14:06:26.876388 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.898357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.898635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.898784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.905681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.905996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.909203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.931568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.932476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.964341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:26.976521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.018294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.018527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.022420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.022665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.091102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.091275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.091473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.091700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.091923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.092143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.093219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.093486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.093755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.093993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.094225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.094476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.117375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.117590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.117797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.117904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.118043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.118137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.118294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.118432 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.118516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.118635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.118715 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.118783 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.118957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.119231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.130649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.131628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.144605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.144831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.145062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.145290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.145512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.145609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.145742 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.145844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.146002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.146115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.146325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.146579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.146803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.147022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 14:06:27.165236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.165447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 14:06:27.165812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.166043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 14:06:27.166143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.166281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 14:06:27.166360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.166565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.166674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.166806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 14:06:27.166897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.167061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.167152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.167326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.167590 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.167845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.178458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.178693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.178964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.179168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.179310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.179479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.179755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.180043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.188036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.188971 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.191566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.191843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.192167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.192484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.192689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.192833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.193050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.193152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.193231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.193309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.193451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.193589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.193813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.193926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.194056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.194202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.194310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.194390 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.194585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.194863 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.195139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.195410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.223091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.224627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.224725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.224996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.226455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.226741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.231282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.231568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.231869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.233128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.233439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.233770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.239690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.240008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.241464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.241793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.245148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.245438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.245740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.246026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.246304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.246413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.246582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.246708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.246852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.246993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.247232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.247313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.247591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.247891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.248186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.248497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.267863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.268142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.269394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.269685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.272778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.274707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.277509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.277857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.279329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.279646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.283313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.285015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.287247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.289029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.291180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.291573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.291855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.292133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.292416 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.292708 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.293044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.293345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.293630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.293959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.294250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.294533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.312957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.313254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.313536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.313831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.315520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.315839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.316122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.316391 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.322130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.325014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.327023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.327276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.327387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.327751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.330542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.330826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.331121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.331417 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.331695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.331779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.331983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.332265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.332547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.332840 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.347787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.347999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.348098 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.348283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.348420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.348588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.348731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.348884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.349015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.349159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.349299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.349442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.349580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.349885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.370585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.371564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.373908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.375802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.377633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.379432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.380055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.381187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.381501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.381832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.381969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.383582 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.383861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.384146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.392239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.393929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.394740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.396612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.399427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.399774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.400075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.400364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.401456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.401797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.402188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.402544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.410785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.411081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.411384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.412367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.412699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.413004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.419981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.420320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.420509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.420789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.421061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.421335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.421573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 14:06:27.421814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.421984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.422106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.422178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.422241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.422431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.422535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.422701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.422841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.422998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.423131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.423261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.423373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.423487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.423587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.423973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.424172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.424536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.424841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.425131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.425412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 14:06:27.464795 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5648737939c0 pid: 80215 (default app) Jun 06 14:06:27.467359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5648737939c0 pid: 80214 (default app) Jun 06 14:06:27.468636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e0712-5803-44f5-979f-3cb23c322739 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:27.468955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e0712-5803-44f5-979f-3cb23c322739 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:27.469270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e0712-5803-44f5-979f-3cb23c322739 None None] PATH_INFO: `/v3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:27.474152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon Jun 6 14:06:27 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:35.728392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7510-43da-49f6-8001-db6af0e52481 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:35.729059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7510-43da-49f6-8001-db6af0e52481 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:35.729059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7510-43da-49f6-8001-db6af0e52481 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:35.733602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:35.738812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:35.739212 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:35.739382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:35.808154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: INFO dbcounter [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] Registered counter for database keystone Jun 06 14:06:35.829886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG oslo_db.sqlalchemy.engines [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80215) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 06 14:06:35.836352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writer thread running {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 06 14:06:36.126659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG passlib.handlers.bcrypt [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80215) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 06 14:06:36.126993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG passlib.handlers.bcrypt [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80215) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 06 14:06:36.182625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:36.184254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:36.310235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 572 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 06 14:06:36.343493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e7510-43da-49f6-8001-db6af0e52481 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:36.420816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: INFO dbcounter [None req-6e0e7510-43da-49f6-8001-db6af0e52481 None None] Registered counter for database keystone Jun 06 14:06:36.440585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG oslo_db.sqlalchemy.engines [None req-6e0e7510-43da-49f6-8001-db6af0e52481 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80214) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 06 14:06:36.446729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writer thread running {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 06 14:06:36.672152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e7510-43da-49f6-8001-db6af0e52481 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:36.680835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ecbcd-2d8c-477f-b1fd-b5820959460e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:36.681513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ecbcd-2d8c-477f-b1fd-b5820959460e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:36.682389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ecbcd-2d8c-477f-b1fd-b5820959460e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:36.682674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ecbcd-2d8c-477f-b1fd-b5820959460e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:36.692908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ecbcd-2d8c-477f-b1fd-b5820959460e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:36.745875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WARNING keystone.common.rbac_enforcer.enforcer [None req-671ecbcd-2d8c-477f-b1fd-b5820959460e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 06 14:06:36.746534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ecbcd-2d8c-477f-b1fd-b5820959460e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:36.756922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Jun 6 14:06:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:06:36.764113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:36.778764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84661856-6eee-4512-8236-c49c30f8d989 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:36.783147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3ef29-b5b4-4bff-8ee6-ff8a6f2553d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:36.784293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3ef29-b5b4-4bff-8ee6-ff8a6f2553d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:36.784583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3ef29-b5b4-4bff-8ee6-ff8a6f2553d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:36.784860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3ef29-b5b4-4bff-8ee6-ff8a6f2553d5 admin admin] PATH_INFO: `/v3/projects/258fdfe108b84b6c82c61b2a2990c11d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:36.786389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3ef29-b5b4-4bff-8ee6-ff8a6f2553d5 admin admin] RBAC: Authorizing `identity:get_project(project_id=258fdfe108b84b6c82c61b2a2990c11d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:36.836182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f0a3ef29-b5b4-4bff-8ee6-ff8a6f2553d5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 06 14:06:36.836524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3ef29-b5b4-4bff-8ee6-ff8a6f2553d5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:36.842881 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Jun 6 14:06:36 2022] GET /identity/v3/projects/258fdfe108b84b6c82c61b2a2990c11d => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:39.105168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c0a76-b0e5-44c5-9382-a8b49f0f5eaf None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:39.105723 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c0a76-b0e5-44c5-9382-a8b49f0f5eaf None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:39.105840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c0a76-b0e5-44c5-9382-a8b49f0f5eaf None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:39.110198 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:06:39.115710 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa1923-4d26-4ded-9231-314176282459 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:39.116084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa1923-4d26-4ded-9231-314176282459 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:39.116259 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa1923-4d26-4ded-9231-314176282459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:39.136261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-66fa1923-4d26-4ded-9231-314176282459 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:39.137821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-66fa1923-4d26-4ded-9231-314176282459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:39.145618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 06 14:06:39.168091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c0a76-b0e5-44c5-9382-a8b49f0f5eaf None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:39.184402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c0a76-b0e5-44c5-9382-a8b49f0f5eaf None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:39.188682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b868b3-3bdb-4c2b-a83a-030ddff8dd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:39.189331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b868b3-3bdb-4c2b-a83a-030ddff8dd5e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:39.189614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b868b3-3bdb-4c2b-a83a-030ddff8dd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:39.189924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b868b3-3bdb-4c2b-a83a-030ddff8dd5e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:39.198647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b868b3-3bdb-4c2b-a83a-030ddff8dd5e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:39.200435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b868b3-3bdb-4c2b-a83a-030ddff8dd5e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:39.209270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Jun 6 14:06:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:06:39.214991 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa1923-4d26-4ded-9231-314176282459 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:39.224077 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa1923-4d26-4ded-9231-314176282459 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:39.228614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31f3b6-a185-44ef-8c1d-2dfbac248f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:39.229652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31f3b6-a185-44ef-8c1d-2dfbac248f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:39.229973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31f3b6-a185-44ef-8c1d-2dfbac248f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:39.230257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31f3b6-a185-44ef-8c1d-2dfbac248f08 admin admin] PATH_INFO: `/v3/users/e2d188b242aa4be3ac73c99d08c98dbe` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:39.231920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31f3b6-a185-44ef-8c1d-2dfbac248f08 admin admin] RBAC: Authorizing `identity:get_user(user_id=e2d188b242aa4be3ac73c99d08c98dbe)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:39.233803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31f3b6-a185-44ef-8c1d-2dfbac248f08 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:39.235374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon Jun 6 14:06:39 2022] GET /identity/v3/users/e2d188b242aa4be3ac73c99d08c98dbe => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:41.624017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4916bc4-07e7-4d2c-aa33-55622bee9372 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:41.624543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4916bc4-07e7-4d2c-aa33-55622bee9372 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:41.624645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4916bc4-07e7-4d2c-aa33-55622bee9372 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:41.625968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:06:41.631097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd29ca-ac17-44a0-8aa2-c65a812e8eba None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:41.631487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd29ca-ac17-44a0-8aa2-c65a812e8eba None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:41.631719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd29ca-ac17-44a0-8aa2-c65a812e8eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:41.646933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc59251-728c-41aa-a454-95e8965df828 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:41.647219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc59251-728c-41aa-a454-95e8965df828 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:41.647496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc59251-728c-41aa-a454-95e8965df828 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:41.648855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:06:41.653354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-40cd29ca-ac17-44a0-8aa2-c65a812e8eba None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:41.654683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-40cd29ca-ac17-44a0-8aa2-c65a812e8eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:41.657473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff7683-7aac-4e5b-a630-5852191dd200 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:41.660902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff7683-7aac-4e5b-a630-5852191dd200 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:41.660902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff7683-7aac-4e5b-a630-5852191dd200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:41.706531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG passlib.handlers.bcrypt [None req-8dff7683-7aac-4e5b-a630-5852191dd200 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80214) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 06 14:06:41.706945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG passlib.handlers.bcrypt [None req-8dff7683-7aac-4e5b-a630-5852191dd200 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80214) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 06 14:06:41.749884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:41.754236 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8dff7683-7aac-4e5b-a630-5852191dd200 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:41.755864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8dff7683-7aac-4e5b-a630-5852191dd200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:41.773561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd29ca-ac17-44a0-8aa2-c65a812e8eba None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:41.798740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:41.870341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:06:41.889043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd29ca-ac17-44a0-8aa2-c65a812e8eba None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:41.892967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff7683-7aac-4e5b-a630-5852191dd200 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:41.924849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8e7b1-0ff8-49f5-8650-f70e3b008e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:41.925653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8e7b1-0ff8-49f5-8650-f70e3b008e4a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:41.925972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8e7b1-0ff8-49f5-8650-f70e3b008e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:41.926254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8e7b1-0ff8-49f5-8650-f70e3b008e4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:41.938440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8e7b1-0ff8-49f5-8650-f70e3b008e4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-8af8e7b1-0ff8-49f5-8650-f70e3b008e4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:06:41.950332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:06:41.951569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:06:41.952518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:06:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:06:41.957382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8e7b1-0ff8-49f5-8650-f70e3b008e4a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:42.000764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff7683-7aac-4e5b-a630-5852191dd200 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:42.003440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:42.041241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:42.041477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765d53f-22a8-4465-9ba4-8cfa63d47d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:42.042359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765d53f-22a8-4465-9ba4-8cfa63d47d68 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:42.042676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765d53f-22a8-4465-9ba4-8cfa63d47d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:42.043070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765d53f-22a8-4465-9ba4-8cfa63d47d68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:42.060997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8e7b1-0ff8-49f5-8650-f70e3b008e4a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:42.094857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed85c9-31e5-453e-aedb-ee068f99a6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:42.095527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed85c9-31e5-453e-aedb-ee068f99a6ed None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:42.095810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed85c9-31e5-453e-aedb-ee068f99a6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:42.096083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed85c9-31e5-453e-aedb-ee068f99a6ed None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:42.096842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed85c9-31e5-453e-aedb-ee068f99a6ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:42.097148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed85c9-31e5-453e-aedb-ee068f99a6ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:42.098604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed85c9-31e5-453e-aedb-ee068f99a6ed None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:42.110892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:06:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:42.113603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765d53f-22a8-4465-9ba4-8cfa63d47d68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:42.115187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765d53f-22a8-4465-9ba4-8cfa63d47d68 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:42.116423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed85c9-31e5-453e-aedb-ee068f99a6ed None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:42.222149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 8/16] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:06:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 14:06:42.224746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed85c9-31e5-453e-aedb-ee068f99a6ed None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:42.227961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765d53f-22a8-4465-9ba4-8cfa63d47d68 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:42.230998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:42.259964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a0d04-e5e2-4460-901e-9f420ecfa7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:42.260941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a0d04-e5e2-4460-901e-9f420ecfa7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:42.261230 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a0d04-e5e2-4460-901e-9f420ecfa7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:42.261507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a0d04-e5e2-4460-901e-9f420ecfa7b0 None admin] PATH_INFO: `/v3/users/e2d188b242aa4be3ac73c99d08c98dbe` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:42.262794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a0d04-e5e2-4460-901e-9f420ecfa7b0 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d188b242aa4be3ac73c99d08c98dbe)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:42.264513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a0d04-e5e2-4460-901e-9f420ecfa7b0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:42.266364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:06:42 2022] GET /identity/v3/users/e2d188b242aa4be3ac73c99d08c98dbe => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:42.274176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a0d04-e5e2-4460-901e-9f420ecfa7b0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:42.289639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:42.336231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765d53f-22a8-4465-9ba4-8cfa63d47d68 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:42.370791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b53f43-07b7-44c2-844a-5a6580aca71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:42.371600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b53f43-07b7-44c2-844a-5a6580aca71d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:42.371887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b53f43-07b7-44c2-844a-5a6580aca71d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:42.372168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b53f43-07b7-44c2-844a-5a6580aca71d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:42.379229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a0d04-e5e2-4460-901e-9f420ecfa7b0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:42.391723 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b53f43-07b7-44c2-844a-5a6580aca71d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:42.393209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b53f43-07b7-44c2-844a-5a6580aca71d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:42.415104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8eb0-313d-4b45-831a-af5d3b49ae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-90b53f43-07b7-44c2-844a-5a6580aca71d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:06:42.415334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:06:42.416214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:06:42.417029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:06:42.417029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:06:42.417029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:06:42.417029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:06:42.417250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8eb0-313d-4b45-831a-af5d3b49ae83 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:42.417250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8eb0-313d-4b45-831a-af5d3b49ae83 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:42.417250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8eb0-313d-4b45-831a-af5d3b49ae83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:42.417426 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:06:42.418544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f8eb0-313d-4b45-831a-af5d3b49ae83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:42.420040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f8eb0-313d-4b45-831a-af5d3b49ae83 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:42.421691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:06:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:42.427090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8eb0-313d-4b45-831a-af5d3b49ae83 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:42.427238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b53f43-07b7-44c2-844a-5a6580aca71d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:42.487594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:42.515654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:42.540359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b53f43-07b7-44c2-844a-5a6580aca71d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:42.541826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8eb0-313d-4b45-831a-af5d3b49ae83 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:42.577649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5f916-9a36-4590-ac37-08bd378c6de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:42.577873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2df4f6-58df-4467-a177-f9e2f8cdec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:42.578354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5f916-9a36-4590-ac37-08bd378c6de3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:42.578490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2df4f6-58df-4467-a177-f9e2f8cdec1c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:42.578640 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2df4f6-58df-4467-a177-f9e2f8cdec1c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:42.578775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5f916-9a36-4590-ac37-08bd378c6de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:42.578920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2df4f6-58df-4467-a177-f9e2f8cdec1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:42.579007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5f916-9a36-4590-ac37-08bd378c6de3 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:42.579650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5f916-9a36-4590-ac37-08bd378c6de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:42.579743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2df4f6-58df-4467-a177-f9e2f8cdec1c None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.domain.id=default, user.id=e2d188b242aa4be3ac73c99d08c98dbe) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:42.579959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5f916-9a36-4590-ac37-08bd378c6de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:42.580044 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2df4f6-58df-4467-a177-f9e2f8cdec1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:42.581368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5f916-9a36-4590-ac37-08bd378c6de3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:42.581492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2df4f6-58df-4467-a177-f9e2f8cdec1c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:42.589272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Mon Jun 6 14:06:42 2022] GET /identity/v3/role_assignments?user.id=e2d188b242aa4be3ac73c99d08c98dbe&scope.domain.id=default&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:42.600917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:45.342096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a5435-8cd9-4abc-ab5d-7ef079304c3f None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:45.342613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a5435-8cd9-4abc-ab5d-7ef079304c3f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:45.342688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a5435-8cd9-4abc-ab5d-7ef079304c3f None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:45.344214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:45.349727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7e893-df94-42b0-8c76-c4ecdaaefb9e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:45.350187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7e893-df94-42b0-8c76-c4ecdaaefb9e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:45.350288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7e893-df94-42b0-8c76-c4ecdaaefb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:45.374993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a5e7e893-df94-42b0-8c76-c4ecdaaefb9e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:45.376211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a5e7e893-df94-42b0-8c76-c4ecdaaefb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:45.455612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25f1b9-73d9-4ca6-8c8e-40cf33dd3441 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:45.455939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25f1b9-73d9-4ca6-8c8e-40cf33dd3441 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:45.456181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25f1b9-73d9-4ca6-8c8e-40cf33dd3441 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:45.457776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 13/23] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:06:45.463494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ab4c5-8b66-42ac-baf5-6228d00b0713 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:45.463812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ab4c5-8b66-42ac-baf5-6228d00b0713 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:45.464074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ab4c5-8b66-42ac-baf5-6228d00b0713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:45.475416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 11/24] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:45.493910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8e5ab4c5-8b66-42ac-baf5-6228d00b0713 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:45.495197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8e5ab4c5-8b66-42ac-baf5-6228d00b0713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:45.519231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7e893-df94-42b0-8c76-c4ecdaaefb9e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:45.635608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:45.637472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 14/25] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:06:45.678552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ab4c5-8b66-42ac-baf5-6228d00b0713 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:45.740401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7e893-df94-42b0-8c76-c4ecdaaefb9e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:45.810141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ec4b-bfe3-45f7-bc1d-ad06016adee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:45.810922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ec4b-bfe3-45f7-bc1d-ad06016adee8 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:45.811379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ec4b-bfe3-45f7-bc1d-ad06016adee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:45.811801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ec4b-bfe3-45f7-bc1d-ad06016adee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:45.812717 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5ec4b-bfe3-45f7-bc1d-ad06016adee8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:45.815332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5ec4b-bfe3-45f7-bc1d-ad06016adee8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:45.823785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:45.882569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ab4c5-8b66-42ac-baf5-6228d00b0713 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:45.947984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5dc3c-27f0-421f-82e1-3e852f919196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:45.948774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5dc3c-27f0-421f-82e1-3e852f919196 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:45.949063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5dc3c-27f0-421f-82e1-3e852f919196 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:45.949340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5dc3c-27f0-421f-82e1-3e852f919196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:45.976057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5dc3c-27f0-421f-82e1-3e852f919196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:45.977585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5dc3c-27f0-421f-82e1-3e852f919196 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:45.980102 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 12/26] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 6 14:06:45 2022] POST /identity/v3/roles => generated 219 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:06:45.987787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19c14e-fe86-4777-af8c-c96a4bd08a0b None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:45.988372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19c14e-fe86-4777-af8c-c96a4bd08a0b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:45.988882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19c14e-fe86-4777-af8c-c96a4bd08a0b None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:45.990998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 13/27] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:45.997183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5703-606b-44d6-b880-57512a32d790 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:45.997840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5703-606b-44d6-b880-57512a32d790 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:45.998351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5703-606b-44d6-b880-57512a32d790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a6c5dc3c-27f0-421f-82e1-3e852f919196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:06:46.013340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:06:46.014283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:06:46.015096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:06:46.015096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:06:46.015096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:06:46.015096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:06:46.024617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 15/28] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:06:46.030994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8c1b5703-606b-44d6-b880-57512a32d790 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:46.031256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5dc3c-27f0-421f-82e1-3e852f919196 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:46.033085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8c1b5703-606b-44d6-b880-57512a32d790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:46.183257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 14/29] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:46.210773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b5703-606b-44d6-b880-57512a32d790 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:46.266578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5dc3c-27f0-421f-82e1-3e852f919196 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:46.290154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:46.358092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:46.377412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a812f-286f-4270-88a4-c6c5f5d015c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:46.378118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a812f-286f-4270-88a4-c6c5f5d015c1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:46.378411 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a812f-286f-4270-88a4-c6c5f5d015c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:46.378688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a812f-286f-4270-88a4-c6c5f5d015c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.379609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a812f-286f-4270-88a4-c6c5f5d015c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:46.380076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a812f-286f-4270-88a4-c6c5f5d015c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:46.381736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a812f-286f-4270-88a4-c6c5f5d015c1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:46.390421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b5703-606b-44d6-b880-57512a32d790 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:46.408748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 16/30] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:46.414629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a812f-286f-4270-88a4-c6c5f5d015c1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:46.437542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfb604-6e7d-4f19-b0a4-aed178291a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:46.438729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb604-6e7d-4f19-b0a4-aed178291a4c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:46.439235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb604-6e7d-4f19-b0a4-aed178291a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:46.439728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb604-6e7d-4f19-b0a4-aed178291a4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.453904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfb604-6e7d-4f19-b0a4-aed178291a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0adfb604-6e7d-4f19-b0a4-aed178291a4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:06:46.464734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:06:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:06:46.466428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:06:46.468894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 15/31] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:06:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:06:46.478814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfb604-6e7d-4f19-b0a4-aed178291a4c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:46.528794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a812f-286f-4270-88a4-c6c5f5d015c1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:46.581864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128f6de-b546-40b7-aca6-fe7755c8d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:46.583620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f6de-b546-40b7-aca6-fe7755c8d479 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:46.584117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f6de-b546-40b7-aca6-fe7755c8d479 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:46.584837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f6de-b546-40b7-aca6-fe7755c8d479 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.586109 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128f6de-b546-40b7-aca6-fe7755c8d479 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:46.589155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128f6de-b546-40b7-aca6-fe7755c8d479 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:46.633235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:46.640234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfb604-6e7d-4f19-b0a4-aed178291a4c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:46.693486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69e137-7600-4b93-89a6-5d316b193a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:46.694641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69e137-7600-4b93-89a6-5d316b193a5e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:46.695173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69e137-7600-4b93-89a6-5d316b193a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:46.695664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69e137-7600-4b93-89a6-5d316b193a5e None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.696785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69e137-7600-4b93-89a6-5d316b193a5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:46.697450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69e137-7600-4b93-89a6-5d316b193a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:46.699731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69e137-7600-4b93-89a6-5d316b193a5e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:46.721352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 16/32] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:06:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:46.723439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4104e9-35dc-4fa1-a010-e0fdaf8a4a53 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:46.724082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4104e9-35dc-4fa1-a010-e0fdaf8a4a53 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:46.724575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4104e9-35dc-4fa1-a010-e0fdaf8a4a53 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.727519 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 17/33] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:46.728902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4104e9-35dc-4fa1-a010-e0fdaf8a4a53 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:46.756419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 17/34] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:06:46 2022] POST /identity/v3/projects => generated 301 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:06:46.758291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be98ac-5d19-42a0-92b0-f0c0669416e4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:46.758590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be98ac-5d19-42a0-92b0-f0c0669416e4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:46.758882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be98ac-5d19-42a0-92b0-f0c0669416e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.778691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f9be98ac-5d19-42a0-92b0-f0c0669416e4 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:46.780317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f9be98ac-5d19-42a0-92b0-f0c0669416e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:46.815882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:46.871145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 18/35] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:46.873568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb494f7-ffaa-4e61-976a-cee2b2d44ae1 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:46.873898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb494f7-ffaa-4e61-976a-cee2b2d44ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:46.874187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb494f7-ffaa-4e61-976a-cee2b2d44ae1 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.875755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 19/36] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:46.876897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4104e9-35dc-4fa1-a010-e0fdaf8a4a53 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:46.883484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0941f52-f297-4132-a794-b059e23081b9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:46.884019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0941f52-f297-4132-a794-b059e23081b9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:46.884531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0941f52-f297-4132-a794-b059e23081b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.908377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b0941f52-f297-4132-a794-b059e23081b9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:46.909910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b0941f52-f297-4132-a794-b059e23081b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:46.917591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0df85-01be-4baa-8ec5-5ad16c451d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:46.918505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0df85-01be-4baa-8ec5-5ad16c451d44 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:46.918791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0df85-01be-4baa-8ec5-5ad16c451d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:46.919075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0df85-01be-4baa-8ec5-5ad16c451d44 None admin] PATH_INFO: `/v3/users/e2d188b242aa4be3ac73c99d08c98dbe` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:46.920905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0df85-01be-4baa-8ec5-5ad16c451d44 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d188b242aa4be3ac73c99d08c98dbe)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:46.922567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0df85-01be-4baa-8ec5-5ad16c451d44 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:46.924757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 18/37] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:06:46 2022] GET /identity/v3/users/e2d188b242aa4be3ac73c99d08c98dbe => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:46.928022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0df85-01be-4baa-8ec5-5ad16c451d44 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:46.960993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:46.998714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 20/38] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:47.006867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0941f52-f297-4132-a794-b059e23081b9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:47.051684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0df85-01be-4baa-8ec5-5ad16c451d44 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:47.098103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02643db6-a604-4595-89b1-9b3f8daa6323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:47.099182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02643db6-a604-4595-89b1-9b3f8daa6323 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:47.099776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02643db6-a604-4595-89b1-9b3f8daa6323 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:47.100256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02643db6-a604-4595-89b1-9b3f8daa6323 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:47.101820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02643db6-a604-4595-89b1-9b3f8daa6323 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:47.103536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02643db6-a604-4595-89b1-9b3f8daa6323 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:47.119106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 19/39] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 6 14:06:46 2022] POST /identity/v3/roles => generated 225 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:06:47.120258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02643db6-a604-4595-89b1-9b3f8daa6323 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:47.140468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:47.145032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0941f52-f297-4132-a794-b059e23081b9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:47.184108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2bb95-f303-4dce-8aa1-345f20ff153e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:47.184934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2bb95-f303-4dce-8aa1-345f20ff153e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:47.185225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2bb95-f303-4dce-8aa1-345f20ff153e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:47.185509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2bb95-f303-4dce-8aa1-345f20ff153e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:47.186874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2bb95-f303-4dce-8aa1-345f20ff153e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:47.188501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2bb95-f303-4dce-8aa1-345f20ff153e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:47.190351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 21/40] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:47.191691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474ed9c-077e-49d0-9be8-da80afbd1f47 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:47.191992 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474ed9c-077e-49d0-9be8-da80afbd1f47 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:47.192276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474ed9c-077e-49d0-9be8-da80afbd1f47 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:47.193907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 22/41] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:47.199830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0e45d-5399-4089-896e-a2350653fcd5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:47.200128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0e45d-5399-4089-896e-a2350653fcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:47.200414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0e45d-5399-4089-896e-a2350653fcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:47.222105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4ed0e45d-5399-4089-896e-a2350653fcd5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:47.224187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4ed0e45d-5399-4089-896e-a2350653fcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:47.230581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:47.252137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02643db6-a604-4595-89b1-9b3f8daa6323 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:47.299914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f48027-e0f2-4785-b0a4-f743d238068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:47.300620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f48027-e0f2-4785-b0a4-f743d238068c None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:47.300951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f48027-e0f2-4785-b0a4-f743d238068c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:47.301187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f48027-e0f2-4785-b0a4-f743d238068c None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:47.302054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f48027-e0f2-4785-b0a4-f743d238068c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:47.303478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f48027-e0f2-4785-b0a4-f743d238068c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:47.312576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 23/42] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:06:47.319285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0e45d-5399-4089-896e-a2350653fcd5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:47.322395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 20/43] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 6 14:06:47 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:06:47.343091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f48027-e0f2-4785-b0a4-f743d238068c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:47.472797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:47.480538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0e45d-5399-4089-896e-a2350653fcd5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:47.503691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f48027-e0f2-4785-b0a4-f743d238068c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:47.526598 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4e128-da4b-4921-9c4c-bb8ae8189821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:47.527639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4e128-da4b-4921-9c4c-bb8ae8189821 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:47.527932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4e128-da4b-4921-9c4c-bb8ae8189821 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:47.528218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4e128-da4b-4921-9c4c-bb8ae8189821 None admin] PATH_INFO: `/v3/domains/default/users/e2d188b242aa4be3ac73c99d08c98dbe/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:47.530774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4e128-da4b-4921-9c4c-bb8ae8189821 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e2d188b242aa4be3ac73c99d08c98dbe, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:47.532287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4e128-da4b-4921-9c4c-bb8ae8189821 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:47.549976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 24/44] 10.222.0.13 () {58 vars in 1361 bytes} [Mon Jun 6 14:06:47 2022] PUT /identity/v3/domains/default/users/e2d188b242aa4be3ac73c99d08c98dbe/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:06:47.553889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de444c5-c01c-40c6-a859-f0ee3e457cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:47.554734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de444c5-c01c-40c6-a859-f0ee3e457cfa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:47.555028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de444c5-c01c-40c6-a859-f0ee3e457cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:47.555312 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de444c5-c01c-40c6-a859-f0ee3e457cfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:47.556614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de444c5-c01c-40c6-a859-f0ee3e457cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:47.558308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de444c5-c01c-40c6-a859-f0ee3e457cfa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:47.560119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 21/45] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:06:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:47.567029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4e128-da4b-4921-9c4c-bb8ae8189821 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:47.687320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4e128-da4b-4921-9c4c-bb8ae8189821 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:47.725020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9201fe-ee3d-4985-beef-ba180d9caa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:47.725727 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9201fe-ee3d-4985-beef-ba180d9caa85 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:47.726040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9201fe-ee3d-4985-beef-ba180d9caa85 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:47.726324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9201fe-ee3d-4985-beef-ba180d9caa85 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:47.727074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9201fe-ee3d-4985-beef-ba180d9caa85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:47.729222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9201fe-ee3d-4985-beef-ba180d9caa85 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:47.759330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 25/46] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:06:47 2022] POST /identity/v3/projects => generated 312 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:06:49.859942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9132e4-4411-4740-a3c7-24e5d994fc24 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:49.860434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9132e4-4411-4740-a3c7-24e5d994fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:49.860434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9132e4-4411-4740-a3c7-24e5d994fc24 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:49.861934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 22/47] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:49.867419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b621c-0b4b-4f51-9584-1ae06911bee9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:49.867843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b621c-0b4b-4f51-9584-1ae06911bee9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:49.868082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b621c-0b4b-4f51-9584-1ae06911bee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:49.884630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-419b621c-0b4b-4f51-9584-1ae06911bee9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:49.886395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-419b621c-0b4b-4f51-9584-1ae06911bee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:49.888536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:49.956689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 26/48] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:49.980538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9132e4-4411-4740-a3c7-24e5d994fc24 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:50.007726 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:50.107032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9132e4-4411-4740-a3c7-24e5d994fc24 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:50.119741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adcf66-1b2b-4dc7-9961-1ceb57ba90b6 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:50.119954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adcf66-1b2b-4dc7-9961-1ceb57ba90b6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:50.120248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adcf66-1b2b-4dc7-9961-1ceb57ba90b6 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:50.121629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 27/49] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:06:50.126671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f566b-9a83-4923-8baf-645f76fcf176 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:50.126960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f566b-9a83-4923-8baf-645f76fcf176 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:50.127242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f566b-9a83-4923-8baf-645f76fcf176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:50.142650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7555c-35ed-446b-9ef1-979b2993c337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:50.143450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7555c-35ed-446b-9ef1-979b2993c337 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:50.143740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7555c-35ed-446b-9ef1-979b2993c337 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:50.144021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7555c-35ed-446b-9ef1-979b2993c337 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:50.144909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-370f566b-9a83-4923-8baf-645f76fcf176 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:50.146200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-370f566b-9a83-4923-8baf-645f76fcf176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:50.154919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7555c-35ed-446b-9ef1-979b2993c337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-02f7555c-35ed-446b-9ef1-979b2993c337 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:06:50.162573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:06:50.163589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:06:50.164416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 23/50] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:06:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:06:50.169815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7555c-35ed-446b-9ef1-979b2993c337 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:50.217609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 28/51] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:06:50.241618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f566b-9a83-4923-8baf-645f76fcf176 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:50.251287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:50.276926 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7555c-35ed-446b-9ef1-979b2993c337 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:50.307418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:50.316026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc814-b59d-4640-9ab4-97bd83825134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:50.316719 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc814-b59d-4640-9ab4-97bd83825134 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:50.317009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc814-b59d-4640-9ab4-97bd83825134 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:50.317294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc814-b59d-4640-9ab4-97bd83825134 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:50.318089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dc814-b59d-4640-9ab4-97bd83825134 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:50.318404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dc814-b59d-4640-9ab4-97bd83825134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:50.319822 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dc814-b59d-4640-9ab4-97bd83825134 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:50.330671 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 24/52] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:50.339056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc814-b59d-4640-9ab4-97bd83825134 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:50.350023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f566b-9a83-4923-8baf-645f76fcf176 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:50.385158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a706ab2-4d4f-4d8a-8b40-52a38cc48818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:50.385874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a706ab2-4d4f-4d8a-8b40-52a38cc48818 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:50.386203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a706ab2-4d4f-4d8a-8b40-52a38cc48818 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:50.386501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a706ab2-4d4f-4d8a-8b40-52a38cc48818 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:50.387777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a706ab2-4d4f-4d8a-8b40-52a38cc48818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:50.389269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a706ab2-4d4f-4d8a-8b40-52a38cc48818 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:50.390947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 29/53] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:06:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:50.396452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a706ab2-4d4f-4d8a-8b40-52a38cc48818 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:50.451257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc814-b59d-4640-9ab4-97bd83825134 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:50.485728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943660ba-8fb7-4a94-9483-557e9c0662c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:50.486661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943660ba-8fb7-4a94-9483-557e9c0662c1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:50.486948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943660ba-8fb7-4a94-9483-557e9c0662c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:50.487235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943660ba-8fb7-4a94-9483-557e9c0662c1 None admin] PATH_INFO: `/v3/users/e2d188b242aa4be3ac73c99d08c98dbe` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:50.488444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943660ba-8fb7-4a94-9483-557e9c0662c1 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d188b242aa4be3ac73c99d08c98dbe)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:50.489952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943660ba-8fb7-4a94-9483-557e9c0662c1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:50.491617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 25/54] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:06:50 2022] GET /identity/v3/users/e2d188b242aa4be3ac73c99d08c98dbe => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:50.497613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943660ba-8fb7-4a94-9483-557e9c0662c1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:50.499129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a706ab2-4d4f-4d8a-8b40-52a38cc48818 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:50.501268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:50.535132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de99102-ca8c-4ceb-85e2-2f3e5db33759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:50.535797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de99102-ca8c-4ceb-85e2-2f3e5db33759 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:50.536085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de99102-ca8c-4ceb-85e2-2f3e5db33759 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:50.536364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de99102-ca8c-4ceb-85e2-2f3e5db33759 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:50.537088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de99102-ca8c-4ceb-85e2-2f3e5db33759 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:50.538574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de99102-ca8c-4ceb-85e2-2f3e5db33759 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:50.543563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:50.565852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 30/55] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Jun 6 14:06:50 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:06:50.611694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943660ba-8fb7-4a94-9483-557e9c0662c1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:50.649832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39c5da-5a7a-422a-97a9-eea747852e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:50.650617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39c5da-5a7a-422a-97a9-eea747852e46 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:50.650908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39c5da-5a7a-422a-97a9-eea747852e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:50.651203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39c5da-5a7a-422a-97a9-eea747852e46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:50.652673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39c5da-5a7a-422a-97a9-eea747852e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:50.654239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39c5da-5a7a-422a-97a9-eea747852e46 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:50.663114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 26/56] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:06:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:06:50.670480 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de99102-ca8c-4ceb-85e2-2f3e5db33759 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:50.805292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de99102-ca8c-4ceb-85e2-2f3e5db33759 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:50.839873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0def03d-a7c5-4e88-a03a-e9c8c0a1c9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:50.840549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0def03d-a7c5-4e88-a03a-e9c8c0a1c9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:50.840843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0def03d-a7c5-4e88-a03a-e9c8c0a1c9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:50.841127 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0def03d-a7c5-4e88-a03a-e9c8c0a1c9d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:50.842008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0def03d-a7c5-4e88-a03a-e9c8c0a1c9d0 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.domain.id=default, user.id=e2d188b242aa4be3ac73c99d08c98dbe) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:50.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0def03d-a7c5-4e88-a03a-e9c8c0a1c9d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:50.843843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0def03d-a7c5-4e88-a03a-e9c8c0a1c9d0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:50.850250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 31/57] 10.222.0.13 () {56 vars in 1322 bytes} [Mon Jun 6 14:06:50 2022] GET /identity/v3/role_assignments?user.id=e2d188b242aa4be3ac73c99d08c98dbe&scope.domain.id=default&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 530 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:52.881912 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0ff58-dc79-4618-91c4-abf240087222 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:52.882427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0ff58-dc79-4618-91c4-abf240087222 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:52.882427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0ff58-dc79-4618-91c4-abf240087222 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:52.883804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 27/58] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:06:52.890718 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c92fa-e1e0-4bd0-82b6-934a25feb42f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:52.891266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c92fa-e1e0-4bd0-82b6-934a25feb42f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:52.891266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c92fa-e1e0-4bd0-82b6-934a25feb42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:52.909762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-752c92fa-e1e0-4bd0-82b6-934a25feb42f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:52.911002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-752c92fa-e1e0-4bd0-82b6-934a25feb42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:52.960470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:52.981414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 32/59] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:53.006261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0ff58-dc79-4618-91c4-abf240087222 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:53.090585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:53.121139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0ff58-dc79-4618-91c4-abf240087222 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:53.154784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7b34-3ca6-4075-b194-95e7fbc9f4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:53.155493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7b34-3ca6-4075-b194-95e7fbc9f4df None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:53.155781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7b34-3ca6-4075-b194-95e7fbc9f4df None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:53.156065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7b34-3ca6-4075-b194-95e7fbc9f4df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:53.157250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba7b34-3ca6-4075-b194-95e7fbc9f4df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:53.158736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba7b34-3ca6-4075-b194-95e7fbc9f4df None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:53.160339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 28/60] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:06:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:53.166721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c92fa-e1e0-4bd0-82b6-934a25feb42f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:53.264852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c92fa-e1e0-4bd0-82b6-934a25feb42f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:53.298821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f03e7-8a54-43e8-860d-d72b77d1c0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:53.299495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f03e7-8a54-43e8-860d-d72b77d1c0f3 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:53.299784 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f03e7-8a54-43e8-860d-d72b77d1c0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:53.300065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f03e7-8a54-43e8-860d-d72b77d1c0f3 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:53.300785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f03e7-8a54-43e8-860d-d72b77d1c0f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:53.302298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f03e7-8a54-43e8-860d-d72b77d1c0f3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:53.327832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 33/61] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:06:53 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:06:56.856345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510a6a-8fcb-4377-825f-d6c6a176da1a None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:56.857619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510a6a-8fcb-4377-825f-d6c6a176da1a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:56.858161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510a6a-8fcb-4377-825f-d6c6a176da1a None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:56.860199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 29/62] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:56.866744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f148985-0c7a-4164-a636-7ebef65b7c26 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:56.867777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f148985-0c7a-4164-a636-7ebef65b7c26 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:56.868415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f148985-0c7a-4164-a636-7ebef65b7c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:56.894432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7f148985-0c7a-4164-a636-7ebef65b7c26 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:56.896110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7f148985-0c7a-4164-a636-7ebef65b7c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:56.924203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b396f68-dcce-4c26-980f-68ce955665fe None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:56.924819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b396f68-dcce-4c26-980f-68ce955665fe None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:56.925341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b396f68-dcce-4c26-980f-68ce955665fe None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:56.926983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 30/63] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:06:56.932939 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903c2ee-661d-411b-9b6a-0cbbe4e4c40d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:56.933551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903c2ee-661d-411b-9b6a-0cbbe4e4c40d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:56.934075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903c2ee-661d-411b-9b6a-0cbbe4e4c40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:56.954555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:56.954921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f903c2ee-661d-411b-9b6a-0cbbe4e4c40d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:56.957463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f903c2ee-661d-411b-9b6a-0cbbe4e4c40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:57.008197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 34/64] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:06:57.009950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598c9b8-f78f-4381-9e44-cd8dc45ec9f4 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.010372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598c9b8-f78f-4381-9e44-cd8dc45ec9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.010783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598c9b8-f78f-4381-9e44-cd8dc45ec9f4 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.012569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 35/65] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:57.018823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdbc7e-e148-4a94-8ea6-d41777a69c1d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.019724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdbc7e-e148-4a94-8ea6-d41777a69c1d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.020152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdbc7e-e148-4a94-8ea6-d41777a69c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.049001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-31fdbc7e-e148-4a94-8ea6-d41777a69c1d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:57.050530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-31fdbc7e-e148-4a94-8ea6-d41777a69c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:57.063499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 31/66] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:57.065130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903c2ee-661d-411b-9b6a-0cbbe4e4c40d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:57.141468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 36/67] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:57.148416 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdbc7e-e148-4a94-8ea6-d41777a69c1d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:57.172243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:57.220653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903c2ee-661d-411b-9b6a-0cbbe4e4c40d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:57.264377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19614a3a-d961-4092-95d4-55c200276daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:57.265193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614a3a-d961-4092-95d4-55c200276daf None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.265321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:57.265488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614a3a-d961-4092-95d4-55c200276daf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.265785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614a3a-d961-4092-95d4-55c200276daf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.270773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdbc7e-e148-4a94-8ea6-d41777a69c1d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:57.278712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19614a3a-d961-4092-95d4-55c200276daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-19614a3a-d961-4092-95d4-55c200276daf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:06:57.286588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:06:57.287506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:06:57.288282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 32/68] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:06:57.288936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19614a3a-d961-4092-95d4-55c200276daf None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:57.316848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad15fb-8b7d-4911-b684-7d17169510ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:57.318037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad15fb-8b7d-4911-b684-7d17169510ce None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.318487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad15fb-8b7d-4911-b684-7d17169510ce None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.318961 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad15fb-8b7d-4911-b684-7d17169510ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.331545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad15fb-8b7d-4911-b684-7d17169510ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-5cad15fb-8b7d-4911-b684-7d17169510ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:06:57.339722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:06:57.340759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:06:57.341748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:06:57.343713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 37/69] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:06:57.345141 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53e8b9-884a-43db-9500-7a77fd60e6f1 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.345432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53e8b9-884a-43db-9500-7a77fd60e6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.345726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53e8b9-884a-43db-9500-7a77fd60e6f1 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.349751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 38/70] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:57.349751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53e8b9-884a-43db-9500-7a77fd60e6f1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:57.409507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19614a3a-d961-4092-95d4-55c200276daf None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:57.452806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:57.459523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3337edb-eb2f-429b-bd14-a0c42fcc6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:57.460446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337edb-eb2f-429b-bd14-a0c42fcc6b52 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.461122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337edb-eb2f-429b-bd14-a0c42fcc6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.461413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337edb-eb2f-429b-bd14-a0c42fcc6b52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.474042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53e8b9-884a-43db-9500-7a77fd60e6f1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:57.475592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3337edb-eb2f-429b-bd14-a0c42fcc6b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c3337edb-eb2f-429b-bd14-a0c42fcc6b52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:06:57.486194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:06:57.487159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:06:57.487961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 33/71] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:06:57.488610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3337edb-eb2f-429b-bd14-a0c42fcc6b52 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:57.526206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4ad3b-bb9a-4629-9f29-9cb991c80f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:57.526907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ad3b-bb9a-4629-9f29-9cb991c80f3c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.527199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ad3b-bb9a-4629-9f29-9cb991c80f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.527484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ad3b-bb9a-4629-9f29-9cb991c80f3c None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.528373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ad3b-bb9a-4629-9f29-9cb991c80f3c None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:57.528688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ad3b-bb9a-4629-9f29-9cb991c80f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:57.530217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ad3b-bb9a-4629-9f29-9cb991c80f3c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:57.540343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 39/72] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:57.542309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc256f6b-09ae-46a5-a415-2d2f7dd44584 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.542640 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc256f6b-09ae-46a5-a415-2d2f7dd44584 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.543181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc256f6b-09ae-46a5-a415-2d2f7dd44584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.568646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-fc256f6b-09ae-46a5-a415-2d2f7dd44584 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:57.570541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fc256f6b-09ae-46a5-a415-2d2f7dd44584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:57.590560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:57.623446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3337edb-eb2f-429b-bd14-a0c42fcc6b52 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:57.649614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 40/73] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:57.650377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc256f6b-09ae-46a5-a415-2d2f7dd44584 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:57.669019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988aa1a-3c2e-48a6-b58a-21ee97d16e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:57.670059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988aa1a-3c2e-48a6-b58a-21ee97d16e32 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.670518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988aa1a-3c2e-48a6-b58a-21ee97d16e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.671212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988aa1a-3c2e-48a6-b58a-21ee97d16e32 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.672627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988aa1a-3c2e-48a6-b58a-21ee97d16e32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:57.672955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988aa1a-3c2e-48a6-b58a-21ee97d16e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:57.674440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988aa1a-3c2e-48a6-b58a-21ee97d16e32 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:57.686012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 34/74] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:57.691301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988aa1a-3c2e-48a6-b58a-21ee97d16e32 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:57.751146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:57.766194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc256f6b-09ae-46a5-a415-2d2f7dd44584 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:57.802668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4e86f-8a94-4a32-a0de-f54ee5952e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:57.803346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4e86f-8a94-4a32-a0de-f54ee5952e24 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.803631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4e86f-8a94-4a32-a0de-f54ee5952e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.803908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4e86f-8a94-4a32-a0de-f54ee5952e24 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.804661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4e86f-8a94-4a32-a0de-f54ee5952e24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:57.804902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988aa1a-3c2e-48a6-b58a-21ee97d16e32 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:57.804994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4e86f-8a94-4a32-a0de-f54ee5952e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:57.806396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4e86f-8a94-4a32-a0de-f54ee5952e24 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:57.814553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 41/75] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:57.815355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4e86f-8a94-4a32-a0de-f54ee5952e24 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:57.829744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:57.844254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d591d-7390-4de8-b6b5-16b3f0995fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:57.845158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d591d-7390-4de8-b6b5-16b3f0995fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.845450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d591d-7390-4de8-b6b5-16b3f0995fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.845811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d591d-7390-4de8-b6b5-16b3f0995fd4 None admin] PATH_INFO: `/v3/users/0e961d4918e44e499e44d50410595f6d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.867531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d591d-7390-4de8-b6b5-16b3f0995fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=0e961d4918e44e499e44d50410595f6d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:57.869097 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d591d-7390-4de8-b6b5-16b3f0995fd4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:57.871531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 35/76] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/users/0e961d4918e44e499e44d50410595f6d => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:57.872396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d591d-7390-4de8-b6b5-16b3f0995fd4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:57.933691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4e86f-8a94-4a32-a0de-f54ee5952e24 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:57.969906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87337ee-c3f1-4c8e-b56b-0a43a1b63c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:57.970736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87337ee-c3f1-4c8e-b56b-0a43a1b63c78 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.971027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87337ee-c3f1-4c8e-b56b-0a43a1b63c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.971320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87337ee-c3f1-4c8e-b56b-0a43a1b63c78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.983465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87337ee-c3f1-4c8e-b56b-0a43a1b63c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:06:57.983666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d591d-7390-4de8-b6b5-16b3f0995fd4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c87337ee-c3f1-4c8e-b56b-0a43a1b63c78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:06:57.991594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:06:57.992734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:06:57.993553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 42/77] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:06:57.994499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f399a0-f239-46b8-bb78-5ca6b7d5934f None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:57.994786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f399a0-f239-46b8-bb78-5ca6b7d5934f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:57.995068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f399a0-f239-46b8-bb78-5ca6b7d5934f None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:57.996458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 43/78] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:06:57.997200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f399a0-f239-46b8-bb78-5ca6b7d5934f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.001090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.026596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da113d-62d0-4809-81ff-620558219ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:58.027485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da113d-62d0-4809-81ff-620558219ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.027768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da113d-62d0-4809-81ff-620558219ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.028047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da113d-62d0-4809-81ff-620558219ed2 None admin] PATH_INFO: `/v3/users/e2d188b242aa4be3ac73c99d08c98dbe` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.029287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da113d-62d0-4809-81ff-620558219ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d188b242aa4be3ac73c99d08c98dbe)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:58.030799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da113d-62d0-4809-81ff-620558219ed2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:58.032470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 36/79] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/users/e2d188b242aa4be3ac73c99d08c98dbe => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:58.033243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da113d-62d0-4809-81ff-620558219ed2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.079062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.111623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f399a0-f239-46b8-bb78-5ca6b7d5934f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:58.140297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da113d-62d0-4809-81ff-620558219ed2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:58.146225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fe7ea-df09-42ad-9bf9-6b03ce2984eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:58.147120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fe7ea-df09-42ad-9bf9-6b03ce2984eb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.147408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fe7ea-df09-42ad-9bf9-6b03ce2984eb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.147683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fe7ea-df09-42ad-9bf9-6b03ce2984eb None admin] PATH_INFO: `/v3/users/0e961d4918e44e499e44d50410595f6d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.149194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fe7ea-df09-42ad-9bf9-6b03ce2984eb None admin] RBAC: Authorizing `identity:get_user(user_id=0e961d4918e44e499e44d50410595f6d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:58.151930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fe7ea-df09-42ad-9bf9-6b03ce2984eb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:58.153662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 44/80] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:06:57 2022] GET /identity/v3/users/0e961d4918e44e499e44d50410595f6d => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:58.154474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fe7ea-df09-42ad-9bf9-6b03ce2984eb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.177515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321701c-0f18-48b8-946f-28ec44c6f332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:58.178204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321701c-0f18-48b8-946f-28ec44c6f332 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.178503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321701c-0f18-48b8-946f-28ec44c6f332 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.178778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321701c-0f18-48b8-946f-28ec44c6f332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.224780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.249125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321701c-0f18-48b8-946f-28ec44c6f332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:58.250640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321701c-0f18-48b8-946f-28ec44c6f332 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:58.257418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fe7ea-df09-42ad-9bf9-6b03ce2984eb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:58.290163 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ec2bc-aa72-4370-978a-0b77b6f98995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:58.290837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ec2bc-aa72-4370-978a-0b77b6f98995 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.291798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ec2bc-aa72-4370-978a-0b77b6f98995 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.292348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ec2bc-aa72-4370-978a-0b77b6f98995 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.292673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ec2bc-aa72-4370-978a-0b77b6f98995 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:58.292981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ec2bc-aa72-4370-978a-0b77b6f98995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:58.294405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ec2bc-aa72-4370-978a-0b77b6f98995 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:58.302844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 45/81] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:06:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:58.303889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a23d53-3c08-4a1c-9d32-0cb1292a4f78 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.304180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a23d53-3c08-4a1c-9d32-0cb1292a4f78 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.304462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a23d53-3c08-4a1c-9d32-0cb1292a4f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.309843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.323552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b7a23d53-3c08-4a1c-9d32-0cb1292a4f78 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:06:58.324789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b7a23d53-3c08-4a1c-9d32-0cb1292a4f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:06:58.354608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 37/82] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:06:58.355579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321701c-0f18-48b8-946f-28ec44c6f332 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.394768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 46/83] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:06:58.395641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a23d53-3c08-4a1c-9d32-0cb1292a4f78 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.436367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.459301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321701c-0f18-48b8-946f-28ec44c6f332 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:58.493600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea6139-58c1-497b-987c-73eeb0fdea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:58.494774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea6139-58c1-497b-987c-73eeb0fdea94 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.495098 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea6139-58c1-497b-987c-73eeb0fdea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.495384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea6139-58c1-497b-987c-73eeb0fdea94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.503927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a23d53-3c08-4a1c-9d32-0cb1292a4f78 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:58.533743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.539595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4feb2-eb36-4b8d-b7ca-19f0532908df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:58.540257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4feb2-eb36-4b8d-b7ca-19f0532908df None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.540548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4feb2-eb36-4b8d-b7ca-19f0532908df None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.540831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4feb2-eb36-4b8d-b7ca-19f0532908df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.564827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea6139-58c1-497b-987c-73eeb0fdea94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:58.566464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea6139-58c1-497b-987c-73eeb0fdea94 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:58.614962 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4feb2-eb36-4b8d-b7ca-19f0532908df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:58.616470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4feb2-eb36-4b8d-b7ca-19f0532908df None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:58.643845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.668318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 38/84] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:06:58.669181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea6139-58c1-497b-987c-73eeb0fdea94 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.720349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 47/85] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:06:58.721213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4feb2-eb36-4b8d-b7ca-19f0532908df None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.752260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.770266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea6139-58c1-497b-987c-73eeb0fdea94 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:58.805244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d124fb-55cb-4d1e-aa86-a42bdbc653f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:58.806158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d124fb-55cb-4d1e-aa86-a42bdbc653f7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.806453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d124fb-55cb-4d1e-aa86-a42bdbc653f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.806807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d124fb-55cb-4d1e-aa86-a42bdbc653f7 None admin] PATH_INFO: `/v3/users/0e961d4918e44e499e44d50410595f6d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.808197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d124fb-55cb-4d1e-aa86-a42bdbc653f7 None admin] RBAC: Authorizing `identity:get_user(user_id=0e961d4918e44e499e44d50410595f6d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:58.809650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d124fb-55cb-4d1e-aa86-a42bdbc653f7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:58.811599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 39/86] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:06:58 2022] GET /identity/v3/users/0e961d4918e44e499e44d50410595f6d => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:58.812157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d124fb-55cb-4d1e-aa86-a42bdbc653f7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.825531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4feb2-eb36-4b8d-b7ca-19f0532908df None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:58.860695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fac97a-8ac9-47ed-add3-3da0fc246e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:58.861400 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.861592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fac97a-8ac9-47ed-add3-3da0fc246e91 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.861913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fac97a-8ac9-47ed-add3-3da0fc246e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.862198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fac97a-8ac9-47ed-add3-3da0fc246e91 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.863538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fac97a-8ac9-47ed-add3-3da0fc246e91 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:58.865000 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fac97a-8ac9-47ed-add3-3da0fc246e91 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:58.873323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 48/87] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:06:58 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:58.873323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fac97a-8ac9-47ed-add3-3da0fc246e91 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.921112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d124fb-55cb-4d1e-aa86-a42bdbc653f7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:58.955503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef63bdf-78e6-4f22-ae99-31245522eef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:58.956305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef63bdf-78e6-4f22-ae99-31245522eef0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:58.956588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef63bdf-78e6-4f22-ae99-31245522eef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:58.957165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef63bdf-78e6-4f22-ae99-31245522eef0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:58.958416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef63bdf-78e6-4f22-ae99-31245522eef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:58.959923 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef63bdf-78e6-4f22-ae99-31245522eef0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:58.961839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 40/88] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:58.962501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef63bdf-78e6-4f22-ae99-31245522eef0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:58.975136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:58.978558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fac97a-8ac9-47ed-add3-3da0fc246e91 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:59.013777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f72527-e409-474b-bbb2-f4bdba93f7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:59.014564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f72527-e409-474b-bbb2-f4bdba93f7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:59.014853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f72527-e409-474b-bbb2-f4bdba93f7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:59.015136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f72527-e409-474b-bbb2-f4bdba93f7f7 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:59.016334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f72527-e409-474b-bbb2-f4bdba93f7f7 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:59.017835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f72527-e409-474b-bbb2-f4bdba93f7f7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:59.019534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 49/89] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:06:58 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:59.020308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f72527-e409-474b-bbb2-f4bdba93f7f7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:59.064543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef63bdf-78e6-4f22-ae99-31245522eef0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:59.079277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:59.100668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711be4d-3289-4469-b1bd-03154b8925ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:59.101523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711be4d-3289-4469-b1bd-03154b8925ae None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:59.101838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711be4d-3289-4469-b1bd-03154b8925ae None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:59.102117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711be4d-3289-4469-b1bd-03154b8925ae None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:59.103307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711be4d-3289-4469-b1bd-03154b8925ae None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:59.105157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711be4d-3289-4469-b1bd-03154b8925ae None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:59.107138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 41/90] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:06:58 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:59.107908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711be4d-3289-4469-b1bd-03154b8925ae None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:59.128809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f72527-e409-474b-bbb2-f4bdba93f7f7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:59.166837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb59ae-ef86-4c0b-8355-b854dcfe4e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:59.167493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb59ae-ef86-4c0b-8355-b854dcfe4e31 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:59.167779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb59ae-ef86-4c0b-8355-b854dcfe4e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:59.168066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb59ae-ef86-4c0b-8355-b854dcfe4e31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:59.205620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:59.219343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711be4d-3289-4469-b1bd-03154b8925ae None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:59.238580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb59ae-ef86-4c0b-8355-b854dcfe4e31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:59.240089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb59ae-ef86-4c0b-8355-b854dcfe4e31 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:59.252886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c67884-905a-44b0-82f3-7eb1e8aeb13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:59.253536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c67884-905a-44b0-82f3-7eb1e8aeb13e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:59.253869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c67884-905a-44b0-82f3-7eb1e8aeb13e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:59.254148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c67884-905a-44b0-82f3-7eb1e8aeb13e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:59.255001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c67884-905a-44b0-82f3-7eb1e8aeb13e None admin] RBAC: Adding query filter params (role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125, user.id=0e961d4918e44e499e44d50410595f6d) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:59.255313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c67884-905a-44b0-82f3-7eb1e8aeb13e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:59.256736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c67884-905a-44b0-82f3-7eb1e8aeb13e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:59.265896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 42/91] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:06:59 2022] GET /identity/v3/role_assignments?user.id=0e961d4918e44e499e44d50410595f6d&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=4a06d59562de4d598671a462e3eaf62a => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:59.266613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c67884-905a-44b0-82f3-7eb1e8aeb13e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:59.320194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:59.354985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 50/92] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:06:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:06:59.356305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb59ae-ef86-4c0b-8355-b854dcfe4e31 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:59.389165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c67884-905a-44b0-82f3-7eb1e8aeb13e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:59.428071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0edd49-b051-41c1-8f96-f4b8f497e3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:59.428796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0edd49-b051-41c1-8f96-f4b8f497e3f8 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:59.429087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0edd49-b051-41c1-8f96-f4b8f497e3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:59.429368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0edd49-b051-41c1-8f96-f4b8f497e3f8 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:59.430141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0edd49-b051-41c1-8f96-f4b8f497e3f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:59.432185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0edd49-b051-41c1-8f96-f4b8f497e3f8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:59.440158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:59.477732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 43/93] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:06:59 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:06:59.477732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0edd49-b051-41c1-8f96-f4b8f497e3f8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:59.481511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb59ae-ef86-4c0b-8355-b854dcfe4e31 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:59.534803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ea170-4cca-4540-829f-76c5f9701667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:59.535980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ea170-4cca-4540-829f-76c5f9701667 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:59.536516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ea170-4cca-4540-829f-76c5f9701667 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:59.537009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ea170-4cca-4540-829f-76c5f9701667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:59.538953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ea170-4cca-4540-829f-76c5f9701667 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125, user.id=e2d188b242aa4be3ac73c99d08c98dbe) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:59.539542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ea170-4cca-4540-829f-76c5f9701667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:59.542559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ea170-4cca-4540-829f-76c5f9701667 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:59.554058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 51/94] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:06:59 2022] GET /identity/v3/role_assignments?user.id=e2d188b242aa4be3ac73c99d08c98dbe&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:59.555532 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ea170-4cca-4540-829f-76c5f9701667 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:59.705530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0edd49-b051-41c1-8f96-f4b8f497e3f8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:59.707776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:59.759707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e276f-fb9e-4b3f-b619-1b4e61239bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:59.761139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e276f-fb9e-4b3f-b619-1b4e61239bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:59.761643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e276f-fb9e-4b3f-b619-1b4e61239bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:59.761984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e276f-fb9e-4b3f-b619-1b4e61239bd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:59.762964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e276f-fb9e-4b3f-b619-1b4e61239bd2 None admin] RBAC: Adding query filter params (role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125, user.id=0e961d4918e44e499e44d50410595f6d) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:06:59.763285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e276f-fb9e-4b3f-b619-1b4e61239bd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:59.764970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e276f-fb9e-4b3f-b619-1b4e61239bd2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:59.772474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 44/95] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:06:59 2022] GET /identity/v3/role_assignments?user.id=0e961d4918e44e499e44d50410595f6d&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:59.775115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ea170-4cca-4540-829f-76c5f9701667 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:06:59.818425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:06:59.832126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456db3f-e181-4e34-8c59-418c2ae3cecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:06:59.833027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456db3f-e181-4e34-8c59-418c2ae3cecc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:06:59.833320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456db3f-e181-4e34-8c59-418c2ae3cecc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:06:59.833602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456db3f-e181-4e34-8c59-418c2ae3cecc None admin] PATH_INFO: `/v3/projects/8239487747da496d8247459b6e4c7051` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:06:59.834927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456db3f-e181-4e34-8c59-418c2ae3cecc None admin] RBAC: Authorizing `identity:get_project(project_id=8239487747da496d8247459b6e4c7051)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:06:59.836414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456db3f-e181-4e34-8c59-418c2ae3cecc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:06:59.838774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 52/96] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:06:59 2022] GET /identity/v3/projects/8239487747da496d8247459b6e4c7051 => generated 312 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:06:59.846902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e276f-fb9e-4b3f-b619-1b4e61239bd2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:06:59.997362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e276f-fb9e-4b3f-b619-1b4e61239bd2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:00.038179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879d7c8-d758-4353-97fd-64e6e31ebb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:00.038848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879d7c8-d758-4353-97fd-64e6e31ebb10 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:00.039135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879d7c8-d758-4353-97fd-64e6e31ebb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:00.039414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879d7c8-d758-4353-97fd-64e6e31ebb10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:00.040264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879d7c8-d758-4353-97fd-64e6e31ebb10 None admin] RBAC: Adding query filter params (role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=8239487747da496d8247459b6e4c7051, user.id=0e961d4918e44e499e44d50410595f6d) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:00.040578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879d7c8-d758-4353-97fd-64e6e31ebb10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:00.042721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879d7c8-d758-4353-97fd-64e6e31ebb10 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:00.051793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 45/97] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:06:59 2022] GET /identity/v3/role_assignments?user.id=0e961d4918e44e499e44d50410595f6d&scope.project.id=8239487747da496d8247459b6e4c7051&role.id=4a06d59562de4d598671a462e3eaf62a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:02.190722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14642c-fdef-40be-8ae7-6e656850070e None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:02.191295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14642c-fdef-40be-8ae7-6e656850070e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:02.191295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14642c-fdef-40be-8ae7-6e656850070e None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:02.192856 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 53/98] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:02.198720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2e1a1-7324-4b8b-a908-1c41c3d3f0c9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:02.199216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2e1a1-7324-4b8b-a908-1c41c3d3f0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:02.199305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2e1a1-7324-4b8b-a908-1c41c3d3f0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:02.222842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-24f2e1a1-7324-4b8b-a908-1c41c3d3f0c9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:02.224069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-24f2e1a1-7324-4b8b-a908-1c41c3d3f0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:02.261727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:02.332655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 46/99] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:02.364589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14642c-fdef-40be-8ae7-6e656850070e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:02.371918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104256c0-a77d-4748-9dcc-b5a70ee3c22f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:02.372256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104256c0-a77d-4748-9dcc-b5a70ee3c22f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:02.372487 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104256c0-a77d-4748-9dcc-b5a70ee3c22f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:02.374045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 47/100] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:02.386472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514910e5-3859-4d00-88f7-aca0d1b50839 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:02.386812 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514910e5-3859-4d00-88f7-aca0d1b50839 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:02.387056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514910e5-3859-4d00-88f7-aca0d1b50839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:02.431007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-514910e5-3859-4d00-88f7-aca0d1b50839 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:02.433604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-514910e5-3859-4d00-88f7-aca0d1b50839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:02.633539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14642c-fdef-40be-8ae7-6e656850070e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:02.671153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 48/101] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:02.708780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81bced-e6a4-4eb7-8714-3364c97c5933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:02.710627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514910e5-3859-4d00-88f7-aca0d1b50839 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:02.714349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81bced-e6a4-4eb7-8714-3364c97c5933 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:02.714883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81bced-e6a4-4eb7-8714-3364c97c5933 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:02.719927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81bced-e6a4-4eb7-8714-3364c97c5933 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:02.759651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81bced-e6a4-4eb7-8714-3364c97c5933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7b81bced-e6a4-4eb7-8714-3364c97c5933 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:02.770125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:02.771082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:02.771864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:02.786697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 54/102] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:02.807786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81bced-e6a4-4eb7-8714-3364c97c5933 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:02.987460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:02.988989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:03.020038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514910e5-3859-4d00-88f7-aca0d1b50839 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:03.098223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81bced-e6a4-4eb7-8714-3364c97c5933 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:03.106680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a25a3-15e9-4717-8a4a-ebc73d14e500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:03.107778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a25a3-15e9-4717-8a4a-ebc73d14e500 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.108071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a25a3-15e9-4717-8a4a-ebc73d14e500 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.108360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a25a3-15e9-4717-8a4a-ebc73d14e500 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.126159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a25a3-15e9-4717-8a4a-ebc73d14e500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-734a25a3-15e9-4717-8a4a-ebc73d14e500 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:03.142455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:03.143408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:03.144191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 49/103] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:07:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:07:03.149654 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a25a3-15e9-4717-8a4a-ebc73d14e500 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:03.191332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3bad9-836e-41e0-bec4-d5ec5c6ce7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:03.193012 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3bad9-836e-41e0-bec4-d5ec5c6ce7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3bad9-836e-41e0-bec4-d5ec5c6ce7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.194127 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3bad9-836e-41e0-bec4-d5ec5c6ce7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.195625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3bad9-836e-41e0-bec4-d5ec5c6ce7a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:03.196249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3bad9-836e-41e0-bec4-d5ec5c6ce7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:03.199360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3bad9-836e-41e0-bec4-d5ec5c6ce7a8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:03.226449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 55/104] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:03.228365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186540f7-1574-43ee-8a35-ce9f0b427441 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.233733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186540f7-1574-43ee-8a35-ce9f0b427441 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.233733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186540f7-1574-43ee-8a35-ce9f0b427441 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.233733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 56/105] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:03.237222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186540f7-1574-43ee-8a35-ce9f0b427441 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:03.383624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a25a3-15e9-4717-8a4a-ebc73d14e500 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:03.408362 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186540f7-1574-43ee-8a35-ce9f0b427441 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:03.438661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:03.453026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf076d-db3d-4a9b-a077-83bc12f177a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:03.453571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e3ff5-e912-499d-abcb-bba51fcfc3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:03.454616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e3ff5-e912-499d-abcb-bba51fcfc3da None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.454907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e3ff5-e912-499d-abcb-bba51fcfc3da None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.455186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e3ff5-e912-499d-abcb-bba51fcfc3da None admin] PATH_INFO: `/v3/users/0e961d4918e44e499e44d50410595f6d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.455904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf076d-db3d-4a9b-a077-83bc12f177a9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.456432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e3ff5-e912-499d-abcb-bba51fcfc3da None admin] RBAC: Authorizing `identity:get_user(user_id=0e961d4918e44e499e44d50410595f6d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:03.456911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf076d-db3d-4a9b-a077-83bc12f177a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.457693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf076d-db3d-4a9b-a077-83bc12f177a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.457973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e3ff5-e912-499d-abcb-bba51fcfc3da None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:03.459733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf076d-db3d-4a9b-a077-83bc12f177a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:03.465920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 57/106] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:03 2022] GET /identity/v3/users/0e961d4918e44e499e44d50410595f6d => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:03.465920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc80523-412e-4de5-9428-78038b3dfc9c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.465920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc80523-412e-4de5-9428-78038b3dfc9c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.465920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc80523-412e-4de5-9428-78038b3dfc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.466334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf076d-db3d-4a9b-a077-83bc12f177a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:03.468872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf076d-db3d-4a9b-a077-83bc12f177a9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:03.482467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 50/107] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:03.484170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583c210-5a4a-41ab-b94a-58bc3de65637 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.484659 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583c210-5a4a-41ab-b94a-58bc3de65637 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.485172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583c210-5a4a-41ab-b94a-58bc3de65637 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.486706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 51/108] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:03.487617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583c210-5a4a-41ab-b94a-58bc3de65637 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:03.495010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cfc80523-412e-4de5-9428-78038b3dfc9c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:03.496528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cfc80523-412e-4de5-9428-78038b3dfc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:03.512197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:03.581762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 58/109] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:03.582676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc80523-412e-4de5-9428-78038b3dfc9c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:03.641187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583c210-5a4a-41ab-b94a-58bc3de65637 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:03.690945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1194de-69e3-4770-8863-683f5bf72127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:03.692532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1194de-69e3-4770-8863-683f5bf72127 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.693108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1194de-69e3-4770-8863-683f5bf72127 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.693678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1194de-69e3-4770-8863-683f5bf72127 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.695139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1194de-69e3-4770-8863-683f5bf72127 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:03.696722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1194de-69e3-4770-8863-683f5bf72127 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:03.699951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 52/110] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:03 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:03.701479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27720f68-f12e-499f-954c-0da216e7a15b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.701897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27720f68-f12e-499f-954c-0da216e7a15b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.702274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27720f68-f12e-499f-954c-0da216e7a15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.724849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-27720f68-f12e-499f-954c-0da216e7a15b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:03.726887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-27720f68-f12e-499f-954c-0da216e7a15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:03.742571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc80523-412e-4de5-9428-78038b3dfc9c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:03.774603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:03.799205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cc61e-2191-45d9-8b8d-b99979086207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:03.800202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cc61e-2191-45d9-8b8d-b99979086207 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.800553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cc61e-2191-45d9-8b8d-b99979086207 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.800892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cc61e-2191-45d9-8b8d-b99979086207 None admin] PATH_INFO: `/v3/users/e2d188b242aa4be3ac73c99d08c98dbe` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.802284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2cc61e-2191-45d9-8b8d-b99979086207 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d188b242aa4be3ac73c99d08c98dbe)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:03.803764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2cc61e-2191-45d9-8b8d-b99979086207 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:03.805686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 59/111] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:03 2022] GET /identity/v3/users/e2d188b242aa4be3ac73c99d08c98dbe => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:03.806334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cc61e-2191-45d9-8b8d-b99979086207 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:03.827081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 53/112] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:03.828624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27720f68-f12e-499f-954c-0da216e7a15b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:03.842295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:03.957132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cc61e-2191-45d9-8b8d-b99979086207 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:03.958389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27720f68-f12e-499f-954c-0da216e7a15b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:03.997345 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232ea43-8598-42dc-aac8-31f1b8c25afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:03.997603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d4479-5704-49f3-83b5-8224e1638dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:03.998623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232ea43-8598-42dc-aac8-31f1b8c25afe None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.998751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d4479-5704-49f3-83b5-8224e1638dfd None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:03.998853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232ea43-8598-42dc-aac8-31f1b8c25afe None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.998935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d4479-5704-49f3-83b5-8224e1638dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:03.999013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232ea43-8598-42dc-aac8-31f1b8c25afe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:03.999095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d4479-5704-49f3-83b5-8224e1638dfd None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/users/0e961d4918e44e499e44d50410595f6d/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.001083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d4479-5704-49f3-83b5-8224e1638dfd None admin] RBAC: Authorizing `identity:create_grant(project_id=574e2d880a9446af80a6bbdfa7f0b125, user_id=0e961d4918e44e499e44d50410595f6d, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:04.002626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d4479-5704-49f3-83b5-8224e1638dfd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:04.012019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232ea43-8598-42dc-aac8-31f1b8c25afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-4232ea43-8598-42dc-aac8-31f1b8c25afe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:04.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:04.022374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 60/113] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:04.024285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e9786-3d85-434b-b67f-aded99877d04 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.026697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e9786-3d85-434b-b67f-aded99877d04 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.026697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e9786-3d85-434b-b67f-aded99877d04 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.027620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 54/114] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:07:03 2022] PUT /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/users/0e961d4918e44e499e44d50410595f6d/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:04.027620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d4479-5704-49f3-83b5-8224e1638dfd None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:04.030760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 61/115] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:04.034885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e9786-3d85-434b-b67f-aded99877d04 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:04.111486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:04.133002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:04.186833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d4479-5704-49f3-83b5-8224e1638dfd None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:04.188648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e9786-3d85-434b-b67f-aded99877d04 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:04.238598 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c26d9c-9b70-4a88-8be1-6c562c62a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:04.239541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c26d9c-9b70-4a88-8be1-6c562c62a2dd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.239891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c26d9c-9b70-4a88-8be1-6c562c62a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.240179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c26d9c-9b70-4a88-8be1-6c562c62a2dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.246611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27765c73-aa02-44bb-8ea9-96da8968a951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:04.248135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27765c73-aa02-44bb-8ea9-96da8968a951 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.248589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27765c73-aa02-44bb-8ea9-96da8968a951 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.249044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27765c73-aa02-44bb-8ea9-96da8968a951 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.251050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27765c73-aa02-44bb-8ea9-96da8968a951 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:04.255267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27765c73-aa02-44bb-8ea9-96da8968a951 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:04.257203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 55/116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:04 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:04.257352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c26d9c-9b70-4a88-8be1-6c562c62a2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:04.258866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e034fb-d771-47d1-a2a7-7a36339e559f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.259338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e034fb-d771-47d1-a2a7-7a36339e559f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.260320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e034fb-d771-47d1-a2a7-7a36339e559f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.266265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 56/117] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d8c26d9c-9b70-4a88-8be1-6c562c62a2dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:04.266704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:04.267600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:04.269444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:04.270144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e034fb-d771-47d1-a2a7-7a36339e559f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:04.270218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 62/118] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:04.270218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45275c-91e8-41bb-857a-4b6012b81077 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.270218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45275c-91e8-41bb-857a-4b6012b81077 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.274137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45275c-91e8-41bb-857a-4b6012b81077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.295830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0a45275c-91e8-41bb-857a-4b6012b81077 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:04.297454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0a45275c-91e8-41bb-857a-4b6012b81077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:04.392946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 63/119] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:04.392946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45275c-91e8-41bb-857a-4b6012b81077 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:04.441369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:04.457904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e034fb-d771-47d1-a2a7-7a36339e559f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:04.517910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8a81-6a42-4b14-997f-66a4d8c06726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:04.519409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8a81-6a42-4b14-997f-66a4d8c06726 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.520137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8a81-6a42-4b14-997f-66a4d8c06726 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.520843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8a81-6a42-4b14-997f-66a4d8c06726 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.522395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8a81-6a42-4b14-997f-66a4d8c06726 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:04.523126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8a81-6a42-4b14-997f-66a4d8c06726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:04.526444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8a81-6a42-4b14-997f-66a4d8c06726 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:04.536819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:04.549963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45275c-91e8-41bb-857a-4b6012b81077 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:04.550510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 57/120] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:04.553005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b58a3b-c494-4ab7-8a54-9cbaf20e2021 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.553493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b58a3b-c494-4ab7-8a54-9cbaf20e2021 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.554777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b58a3b-c494-4ab7-8a54-9cbaf20e2021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.581643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-29b58a3b-c494-4ab7-8a54-9cbaf20e2021 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:04.583656 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-29b58a3b-c494-4ab7-8a54-9cbaf20e2021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:04.622242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63a40f-81aa-420f-91ee-ef2ee90a514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:04.623700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63a40f-81aa-420f-91ee-ef2ee90a514b None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.624136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63a40f-81aa-420f-91ee-ef2ee90a514b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.624609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63a40f-81aa-420f-91ee-ef2ee90a514b None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/users/e2d188b242aa4be3ac73c99d08c98dbe/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.627706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63a40f-81aa-420f-91ee-ef2ee90a514b None admin] RBAC: Authorizing `identity:create_grant(project_id=574e2d880a9446af80a6bbdfa7f0b125, user_id=e2d188b242aa4be3ac73c99d08c98dbe, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:04.630146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63a40f-81aa-420f-91ee-ef2ee90a514b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:04.648650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 64/121] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:07:04 2022] PUT /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/users/e2d188b242aa4be3ac73c99d08c98dbe/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:04.650112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63a40f-81aa-420f-91ee-ef2ee90a514b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:04.704733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 58/122] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:04.706823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b58a3b-c494-4ab7-8a54-9cbaf20e2021 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:04.873809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:04.880617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63a40f-81aa-420f-91ee-ef2ee90a514b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:04.906237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b58a3b-c494-4ab7-8a54-9cbaf20e2021 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:04.956602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5e26b-636a-4b47-8a5b-a6ce74b74372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:04.958255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5e26b-636a-4b47-8a5b-a6ce74b74372 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.959020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5e26b-636a-4b47-8a5b-a6ce74b74372 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.959775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5e26b-636a-4b47-8a5b-a6ce74b74372 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.962036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5e26b-636a-4b47-8a5b-a6ce74b74372 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:04.962993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5e26b-636a-4b47-8a5b-a6ce74b74372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:04.966036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5e26b-636a-4b47-8a5b-a6ce74b74372 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:04.966264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:04.980794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5e653-5945-4446-9ff8-c830794c72d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:04.982352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e653-5945-4446-9ff8-c830794c72d7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:04.982819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e653-5945-4446-9ff8-c830794c72d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:04.983305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 65/123] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:04.983648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e653-5945-4446-9ff8-c830794c72d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:04.984790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5e26b-636a-4b47-8a5b-a6ce74b74372 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:05.004315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5e653-5945-4446-9ff8-c830794c72d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c5d5e653-5945-4446-9ff8-c830794c72d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:05.020700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:05.022409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 59/124] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:05.024769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5e653-5945-4446-9ff8-c830794c72d7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:05.106743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5e26b-636a-4b47-8a5b-a6ce74b74372 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:05.151488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986705a-cf33-4279-abea-6f7ce2a1d9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:05.152397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986705a-cf33-4279-abea-6f7ce2a1d9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:05.152766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986705a-cf33-4279-abea-6f7ce2a1d9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:05.153129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986705a-cf33-4279-abea-6f7ce2a1d9d5 None admin] PATH_INFO: `/v3/users/0e961d4918e44e499e44d50410595f6d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:05.154557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986705a-cf33-4279-abea-6f7ce2a1d9d5 None admin] RBAC: Authorizing `identity:get_user(user_id=0e961d4918e44e499e44d50410595f6d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:05.156147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986705a-cf33-4279-abea-6f7ce2a1d9d5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:05.158470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 66/125] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:04 2022] GET /identity/v3/users/0e961d4918e44e499e44d50410595f6d => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:05.159424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986705a-cf33-4279-abea-6f7ce2a1d9d5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:05.200540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:05.221687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5e653-5945-4446-9ff8-c830794c72d7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:05.290484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af192c85-3628-41e6-9e22-6e9b0e009fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:05.291883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af192c85-3628-41e6-9e22-6e9b0e009fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:05.292327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af192c85-3628-41e6-9e22-6e9b0e009fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:05.292874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af192c85-3628-41e6-9e22-6e9b0e009fa4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:05.294827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af192c85-3628-41e6-9e22-6e9b0e009fa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:05.296104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986705a-cf33-4279-abea-6f7ce2a1d9d5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:05.297153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af192c85-3628-41e6-9e22-6e9b0e009fa4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:05.299982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 60/126] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:07:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:05.301150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af192c85-3628-41e6-9e22-6e9b0e009fa4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:05.361969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1b0fb-e958-4117-b601-e04df0794daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:05.363758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1b0fb-e958-4117-b601-e04df0794daa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:05.364418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1b0fb-e958-4117-b601-e04df0794daa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:05.365026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1b0fb-e958-4117-b601-e04df0794daa None admin] PATH_INFO: `/v3/users/506e319379084a9db24f03bbe0c0e2be` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:05.392928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1b0fb-e958-4117-b601-e04df0794daa None admin] RBAC: Authorizing `identity:get_user(user_id=506e319379084a9db24f03bbe0c0e2be)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:05.395975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1b0fb-e958-4117-b601-e04df0794daa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:05.405997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 67/127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:05 2022] GET /identity/v3/users/506e319379084a9db24f03bbe0c0e2be => generated 289 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:05.407688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1b0fb-e958-4117-b601-e04df0794daa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:05.450165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:05.508012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af192c85-3628-41e6-9e22-6e9b0e009fa4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:05.573290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c99a2-6882-48e5-b388-a7f6ef082b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:05.574500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c99a2-6882-48e5-b388-a7f6ef082b49 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:05.574999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c99a2-6882-48e5-b388-a7f6ef082b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:05.575766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c99a2-6882-48e5-b388-a7f6ef082b49 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:05.577324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c99a2-6882-48e5-b388-a7f6ef082b49 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:05.577915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c99a2-6882-48e5-b388-a7f6ef082b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:05.580490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c99a2-6882-48e5-b388-a7f6ef082b49 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:05.596686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:05.599250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 61/128] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:05.600885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c99a2-6882-48e5-b388-a7f6ef082b49 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:05.632720 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1b0fb-e958-4117-b601-e04df0794daa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:05.694678 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f8f49-064c-484b-ab72-2b9d36f5f301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:05.695462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f8f49-064c-484b-ab72-2b9d36f5f301 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:05.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f8f49-064c-484b-ab72-2b9d36f5f301 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:05.696018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f8f49-064c-484b-ab72-2b9d36f5f301 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:05.699994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f8f49-064c-484b-ab72-2b9d36f5f301 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:05.699994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f8f49-064c-484b-ab72-2b9d36f5f301 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:05.702775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 68/129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:05 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:05.702775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f8f49-064c-484b-ab72-2b9d36f5f301 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:05.799905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c99a2-6882-48e5-b388-a7f6ef082b49 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:05.837775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:05.866609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce68223-4b45-4ad9-a335-ff60b6faa443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:05.867784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce68223-4b45-4ad9-a335-ff60b6faa443 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:05.868288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce68223-4b45-4ad9-a335-ff60b6faa443 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:05.868798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce68223-4b45-4ad9-a335-ff60b6faa443 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:05.870497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce68223-4b45-4ad9-a335-ff60b6faa443 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:05.873341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce68223-4b45-4ad9-a335-ff60b6faa443 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:05.897201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f8f49-064c-484b-ab72-2b9d36f5f301 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:05.927961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 62/130] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:07:05 2022] POST /identity/v3/projects => generated 302 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:07:05.929646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce68223-4b45-4ad9-a335-ff60b6faa443 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:05.971886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d37f1-c5e6-4201-bbbb-b417dcdca679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:05.973545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d37f1-c5e6-4201-bbbb-b417dcdca679 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:05.974454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d37f1-c5e6-4201-bbbb-b417dcdca679 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:05.975333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d37f1-c5e6-4201-bbbb-b417dcdca679 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:06.032047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:06.148843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce68223-4b45-4ad9-a335-ff60b6faa443 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:06.169830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d37f1-c5e6-4201-bbbb-b417dcdca679 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:06.171488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d37f1-c5e6-4201-bbbb-b417dcdca679 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:06.248240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e8a3b-f6eb-4219-8663-d579d7f103a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:06.249873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e8a3b-f6eb-4219-8663-d579d7f103a4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:06.250379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e8a3b-f6eb-4219-8663-d579d7f103a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:06.250874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e8a3b-f6eb-4219-8663-d579d7f103a4 None admin] PATH_INFO: `/v3/users/0e961d4918e44e499e44d50410595f6d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:06.256990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e8a3b-f6eb-4219-8663-d579d7f103a4 None admin] RBAC: Authorizing `identity:get_user(user_id=0e961d4918e44e499e44d50410595f6d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:06.259984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e8a3b-f6eb-4219-8663-d579d7f103a4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:06.271649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 63/131] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:05 2022] GET /identity/v3/users/0e961d4918e44e499e44d50410595f6d => generated 275 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:06.272595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e8a3b-f6eb-4219-8663-d579d7f103a4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:06.330227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:06.402394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 69/132] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:06.403316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d37f1-c5e6-4201-bbbb-b417dcdca679 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:06.405967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e8a3b-f6eb-4219-8663-d579d7f103a4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:06.458097 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd7876-1a2a-431f-b66a-699da3be70ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:06.459030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd7876-1a2a-431f-b66a-699da3be70ae None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:06.459326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd7876-1a2a-431f-b66a-699da3be70ae None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:06.459612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd7876-1a2a-431f-b66a-699da3be70ae None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/users/0e961d4918e44e499e44d50410595f6d/roles/49053ad44c6c4cd1b932fcdb45321ff9` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:06.464438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd7876-1a2a-431f-b66a-699da3be70ae None admin] RBAC: Authorizing `identity:create_grant(project_id=574e2d880a9446af80a6bbdfa7f0b125, user_id=0e961d4918e44e499e44d50410595f6d, role_id=49053ad44c6c4cd1b932fcdb45321ff9)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:06.467267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd7876-1a2a-431f-b66a-699da3be70ae None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:06.486853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 64/133] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:07:06 2022] PUT /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/users/0e961d4918e44e499e44d50410595f6d/roles/49053ad44c6c4cd1b932fcdb45321ff9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:06.487836 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd7876-1a2a-431f-b66a-699da3be70ae None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:06.498126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:06.626111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:06.631485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd7876-1a2a-431f-b66a-699da3be70ae None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:06.659859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d37f1-c5e6-4201-bbbb-b417dcdca679 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:06.682987 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901435e-46a5-4c8a-a8f1-0690046c0b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:06.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901435e-46a5-4c8a-a8f1-0690046c0b79 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:06.685164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901435e-46a5-4c8a-a8f1-0690046c0b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:06.685608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901435e-46a5-4c8a-a8f1-0690046c0b79 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:06.687353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901435e-46a5-4c8a-a8f1-0690046c0b79 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:06.689020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901435e-46a5-4c8a-a8f1-0690046c0b79 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:06.691329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 65/134] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:06 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:06.703186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901435e-46a5-4c8a-a8f1-0690046c0b79 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:06.807534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c437f-180b-4c9f-b2a6-516eab69d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:06.809120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c437f-180b-4c9f-b2a6-516eab69d61b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:06.809356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c437f-180b-4c9f-b2a6-516eab69d61b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:06.809824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c437f-180b-4c9f-b2a6-516eab69d61b None admin] PATH_INFO: `/v3/projects/8239487747da496d8247459b6e4c7051` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:06.812003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c437f-180b-4c9f-b2a6-516eab69d61b None admin] RBAC: Authorizing `identity:get_project(project_id=8239487747da496d8247459b6e4c7051)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:06.813680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c437f-180b-4c9f-b2a6-516eab69d61b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:06.815542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 70/135] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:06 2022] GET /identity/v3/projects/8239487747da496d8247459b6e4c7051 => generated 312 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:06.826182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c437f-180b-4c9f-b2a6-516eab69d61b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:06.854230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901435e-46a5-4c8a-a8f1-0690046c0b79 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:06.902107 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223011c-f8e7-435a-bbd8-280d57bbf467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:06.903356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223011c-f8e7-435a-bbd8-280d57bbf467 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:06.903802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223011c-f8e7-435a-bbd8-280d57bbf467 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:06.904861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223011c-f8e7-435a-bbd8-280d57bbf467 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:06.906274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223011c-f8e7-435a-bbd8-280d57bbf467 None admin] RBAC: Adding query filter params (role.id=443ba7d155f54b0f85e995b760bd77bb, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125, user.id=506e319379084a9db24f03bbe0c0e2be) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:06.906779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223011c-f8e7-435a-bbd8-280d57bbf467 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:06.908738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223011c-f8e7-435a-bbd8-280d57bbf467 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:06.921483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 66/136] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:06 2022] GET /identity/v3/role_assignments?user.id=506e319379084a9db24f03bbe0c0e2be&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=443ba7d155f54b0f85e995b760bd77bb => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:07.084220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c437f-180b-4c9f-b2a6-516eab69d61b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:07.129896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:07.165287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e82d2-6c9c-4e17-ba3f-c764fc1f2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:07.168221 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e82d2-6c9c-4e17-ba3f-c764fc1f2dc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:07.168857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e82d2-6c9c-4e17-ba3f-c764fc1f2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:07.169430 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e82d2-6c9c-4e17-ba3f-c764fc1f2dc5 None admin] PATH_INFO: `/v3/projects/8239487747da496d8247459b6e4c7051/users/0e961d4918e44e499e44d50410595f6d/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:07.173415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e82d2-6c9c-4e17-ba3f-c764fc1f2dc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8239487747da496d8247459b6e4c7051, user_id=0e961d4918e44e499e44d50410595f6d, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:07.176352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e82d2-6c9c-4e17-ba3f-c764fc1f2dc5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:07.222316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 71/137] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:07:06 2022] PUT /identity/v3/projects/8239487747da496d8247459b6e4c7051/users/0e961d4918e44e499e44d50410595f6d/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:07.252686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57db1b3-b118-4702-8de8-f8f317136c2f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:07.253265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57db1b3-b118-4702-8de8-f8f317136c2f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:07.253738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57db1b3-b118-4702-8de8-f8f317136c2f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:07.255423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 67/138] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:07.272532 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceefc65-da02-4dba-8456-a37c5a48263b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:07.273129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceefc65-da02-4dba-8456-a37c5a48263b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:07.273651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceefc65-da02-4dba-8456-a37c5a48263b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:07.312447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9ceefc65-da02-4dba-8456-a37c5a48263b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:07.314866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9ceefc65-da02-4dba-8456-a37c5a48263b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:07.494834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 72/139] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:07.562889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57db1b3-b118-4702-8de8-f8f317136c2f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:07.612949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:07.721232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57db1b3-b118-4702-8de8-f8f317136c2f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:07.781289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebfdeb-ea2a-4322-88b2-9e307b5dd979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:07.782151 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebfdeb-ea2a-4322-88b2-9e307b5dd979 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:07.782436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebfdeb-ea2a-4322-88b2-9e307b5dd979 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:07.782710 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebfdeb-ea2a-4322-88b2-9e307b5dd979 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:07.801353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebfdeb-ea2a-4322-88b2-9e307b5dd979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7bebfdeb-ea2a-4322-88b2-9e307b5dd979 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:07.809684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:07.810689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:07.811481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 68/140] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:07.824344 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceefc65-da02-4dba-8456-a37c5a48263b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:08.096835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceefc65-da02-4dba-8456-a37c5a48263b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:08.127233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:08.180946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c0930-13b2-4c1b-94ce-ac72c6704b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:08.182695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c0930-13b2-4c1b-94ce-ac72c6704b39 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:08.183156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c0930-13b2-4c1b-94ce-ac72c6704b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:08.183680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c0930-13b2-4c1b-94ce-ac72c6704b39 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:08.185113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c0930-13b2-4c1b-94ce-ac72c6704b39 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:08.185656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c0930-13b2-4c1b-94ce-ac72c6704b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:08.188337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c0930-13b2-4c1b-94ce-ac72c6704b39 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:08.211781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 73/141] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:08.218889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebfdeb-ea2a-4322-88b2-9e307b5dd979 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:08.440159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:08.458921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebfdeb-ea2a-4322-88b2-9e307b5dd979 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:08.563579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a198a-eec5-4a66-9708-059bd9446911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:08.565168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a198a-eec5-4a66-9708-059bd9446911 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:08.565673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a198a-eec5-4a66-9708-059bd9446911 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:08.566237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a198a-eec5-4a66-9708-059bd9446911 None admin] PATH_INFO: `/v3/users/0e961d4918e44e499e44d50410595f6d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:08.568946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a198a-eec5-4a66-9708-059bd9446911 None admin] RBAC: Authorizing `identity:get_user(user_id=0e961d4918e44e499e44d50410595f6d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:08.572065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a198a-eec5-4a66-9708-059bd9446911 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:08.576120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 69/142] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:08 2022] GET /identity/v3/users/0e961d4918e44e499e44d50410595f6d => generated 275 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:08.585665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c0930-13b2-4c1b-94ce-ac72c6704b39 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:08.792301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7916c-9d2a-43e5-954f-ce897af788f9 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:08.793037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7916c-9d2a-43e5-954f-ce897af788f9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:08.793321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7916c-9d2a-43e5-954f-ce897af788f9 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:08.800271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 70/143] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:08.808929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06322d8-8b38-4c32-b043-54c541285db9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:08.809373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06322d8-8b38-4c32-b043-54c541285db9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:08.809844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06322d8-8b38-4c32-b043-54c541285db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:08.846046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c0930-13b2-4c1b-94ce-ac72c6704b39 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:08.858084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f06322d8-8b38-4c32-b043-54c541285db9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:08.860229 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f06322d8-8b38-4c32-b043-54c541285db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:08.933340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e4d2c-81ab-44bf-9993-cac394e233d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:08.934556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e4d2c-81ab-44bf-9993-cac394e233d6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:08.935600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e4d2c-81ab-44bf-9993-cac394e233d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:08.936240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e4d2c-81ab-44bf-9993-cac394e233d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:09.038416 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:09.067943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 71/144] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:09.111803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e4d2c-81ab-44bf-9993-cac394e233d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:09.114602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e4d2c-81ab-44bf-9993-cac394e233d6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:09.138336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06322d8-8b38-4c32-b043-54c541285db9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:09.433469 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:09.453210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 74/145] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 869 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:09.460220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e4d2c-81ab-44bf-9993-cac394e233d6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:09.467993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06322d8-8b38-4c32-b043-54c541285db9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:09.608132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3f155-36e7-4009-896a-ebbcf4740085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:09.608433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3f155-36e7-4009-896a-ebbcf4740085 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:09.608920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3f155-36e7-4009-896a-ebbcf4740085 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:09.609374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3f155-36e7-4009-896a-ebbcf4740085 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:09.626118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:09.645250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3f155-36e7-4009-896a-ebbcf4740085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d3a3f155-36e7-4009-896a-ebbcf4740085 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:09.661121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:09.662106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:09.662874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 72/146] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 526 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:07:09.670393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3f155-36e7-4009-896a-ebbcf4740085 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:09.673804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e4d2c-81ab-44bf-9993-cac394e233d6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:09.770198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96db464-962e-4e1a-8ee5-77a216b2227b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:09.771026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96db464-962e-4e1a-8ee5-77a216b2227b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:09.771310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96db464-962e-4e1a-8ee5-77a216b2227b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:09.771589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96db464-962e-4e1a-8ee5-77a216b2227b None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:09.772996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96db464-962e-4e1a-8ee5-77a216b2227b None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:09.774563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96db464-962e-4e1a-8ee5-77a216b2227b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:09.781638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 75/147] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:09 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:09.785341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96db464-962e-4e1a-8ee5-77a216b2227b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:10.003937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96db464-962e-4e1a-8ee5-77a216b2227b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:10.028550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3f155-36e7-4009-896a-ebbcf4740085 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:10.079382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25153651-6421-4950-9b88-72cc88980124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:10.080068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25153651-6421-4950-9b88-72cc88980124 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:10.080359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25153651-6421-4950-9b88-72cc88980124 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:10.080642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25153651-6421-4950-9b88-72cc88980124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:10.081500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25153651-6421-4950-9b88-72cc88980124 None admin] RBAC: Adding query filter params (role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125, user.id=0e961d4918e44e499e44d50410595f6d) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:10.081897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25153651-6421-4950-9b88-72cc88980124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:10.083831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25153651-6421-4950-9b88-72cc88980124 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:10.095562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 76/148] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:09 2022] GET /identity/v3/role_assignments?user.id=0e961d4918e44e499e44d50410595f6d&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=4a06d59562de4d598671a462e3eaf62a => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:10.132379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049bc89-c31e-4774-b2e2-cca6898996d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:10.133083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049bc89-c31e-4774-b2e2-cca6898996d6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:10.133367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049bc89-c31e-4774-b2e2-cca6898996d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:10.133647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049bc89-c31e-4774-b2e2-cca6898996d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:10.134489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049bc89-c31e-4774-b2e2-cca6898996d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:10.135340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049bc89-c31e-4774-b2e2-cca6898996d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:10.137534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049bc89-c31e-4774-b2e2-cca6898996d6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:10.154173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 73/149] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 484 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:10.159873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25153651-6421-4950-9b88-72cc88980124 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:10.174303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:10.327054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25153651-6421-4950-9b88-72cc88980124 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:10.378662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01072d4-2960-4ffb-975b-8c24de25b7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:10.380100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01072d4-2960-4ffb-975b-8c24de25b7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:10.380661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01072d4-2960-4ffb-975b-8c24de25b7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:10.381172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01072d4-2960-4ffb-975b-8c24de25b7b4 None admin] PATH_INFO: `/v3/users/e2d188b242aa4be3ac73c99d08c98dbe` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:10.382972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01072d4-2960-4ffb-975b-8c24de25b7b4 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d188b242aa4be3ac73c99d08c98dbe)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:10.385414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01072d4-2960-4ffb-975b-8c24de25b7b4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:10.388128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 77/150] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:10 2022] GET /identity/v3/users/e2d188b242aa4be3ac73c99d08c98dbe => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:10.394288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049bc89-c31e-4774-b2e2-cca6898996d6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:10.423836 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:10.531385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce817fc-af97-42ab-a1ff-1ec70e6ac295 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:10.532765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce817fc-af97-42ab-a1ff-1ec70e6ac295 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:10.533336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce817fc-af97-42ab-a1ff-1ec70e6ac295 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:10.535146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 78/151] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:10.548529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d383de-feb2-4ea0-b370-a85b4f3613d4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:10.549179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d383de-feb2-4ea0-b370-a85b4f3613d4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:10.549664 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d383de-feb2-4ea0-b370-a85b4f3613d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:10.581473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c5d383de-feb2-4ea0-b370-a85b4f3613d4 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:10.583417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c5d383de-feb2-4ea0-b370-a85b4f3613d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:10.600789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049bc89-c31e-4774-b2e2-cca6898996d6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:10.643014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b124b53-34ad-4db1-93a2-ba1f932639ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:10.643913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b124b53-34ad-4db1-93a2-ba1f932639ae None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:10.644360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b124b53-34ad-4db1-93a2-ba1f932639ae None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:10.644767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b124b53-34ad-4db1-93a2-ba1f932639ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:10.686540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 79/152] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:10.687937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3e47a-114b-4af6-825a-3798a0409aba None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:10.688222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3e47a-114b-4af6-825a-3798a0409aba None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:10.688563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3e47a-114b-4af6-825a-3798a0409aba None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:10.699265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 80/153] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:10.704994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8a6fe-b16d-4feb-b5c8-fee57f83ebb8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:10.705307 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8a6fe-b16d-4feb-b5c8-fee57f83ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:10.705586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8a6fe-b16d-4feb-b5c8-fee57f83ebb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:10.716615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:10.726855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b124b53-34ad-4db1-93a2-ba1f932639ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:10.728776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b124b53-34ad-4db1-93a2-ba1f932639ae None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:10.742213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-edd8a6fe-b16d-4feb-b5c8-fee57f83ebb8 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:10.743583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-edd8a6fe-b16d-4feb-b5c8-fee57f83ebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:10.746164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:10.832351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 81/154] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:10.833490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8a6fe-b16d-4feb-b5c8-fee57f83ebb8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:10.849826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 74/155] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:10.856804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b124b53-34ad-4db1-93a2-ba1f932639ae None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:11.011093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8a6fe-b16d-4feb-b5c8-fee57f83ebb8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:11.018310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b124b53-34ad-4db1-93a2-ba1f932639ae None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:11.036748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:11.058763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9eed27-4c1f-4f58-babf-7c8d4d8ecf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:11.060192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9eed27-4c1f-4f58-babf-7c8d4d8ecf4f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.060914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9eed27-4c1f-4f58-babf-7c8d4d8ecf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.061842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9eed27-4c1f-4f58-babf-7c8d4d8ecf4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.063762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9eed27-4c1f-4f58-babf-7c8d4d8ecf4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:11.065739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9eed27-4c1f-4f58-babf-7c8d4d8ecf4f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:11.066317 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb942e0a-4982-4bfb-bfed-393db43e7f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:11.067372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb942e0a-4982-4bfb-bfed-393db43e7f74 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.067973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 82/156] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:07:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:11.073655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb942e0a-4982-4bfb-bfed-393db43e7f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.073879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9eed27-4c1f-4f58-babf-7c8d4d8ecf4f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:11.074549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb942e0a-4982-4bfb-bfed-393db43e7f74 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.075184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:11.076733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb942e0a-4982-4bfb-bfed-393db43e7f74 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:11.079314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb942e0a-4982-4bfb-bfed-393db43e7f74 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:11.083124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 75/157] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:10 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:11.084802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb942e0a-4982-4bfb-bfed-393db43e7f74 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:11.257626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9eed27-4c1f-4f58-babf-7c8d4d8ecf4f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:11.265986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb942e0a-4982-4bfb-bfed-393db43e7f74 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:11.299410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750e4a7-7831-412c-82a6-dd001956b0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:11.300229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750e4a7-7831-412c-82a6-dd001956b0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.300513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750e4a7-7831-412c-82a6-dd001956b0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.300789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750e4a7-7831-412c-82a6-dd001956b0b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.306748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7c7b8-47d5-40d9-8620-15aaa95b6679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:11.308120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7c7b8-47d5-40d9-8620-15aaa95b6679 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.308683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7c7b8-47d5-40d9-8620-15aaa95b6679 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.309168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7c7b8-47d5-40d9-8620-15aaa95b6679 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.310148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7c7b8-47d5-40d9-8620-15aaa95b6679 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:11.311817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7c7b8-47d5-40d9-8620-15aaa95b6679 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:11.312855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750e4a7-7831-412c-82a6-dd001956b0b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2750e4a7-7831-412c-82a6-dd001956b0b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:11.322035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:11.323046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:11.323878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:11.324969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 83/158] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:11.325804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750e4a7-7831-412c-82a6-dd001956b0b6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:11.339783 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 76/159] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:07:11 2022] POST /identity/v3/users => generated 283 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:07:11.341241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3c55e-dcf8-4941-b551-7db721c0c6f5 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.341597 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3c55e-dcf8-4941-b551-7db721c0c6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.341923 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3c55e-dcf8-4941-b551-7db721c0c6f5 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.343216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 77/160] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:11.343957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3c55e-dcf8-4941-b551-7db721c0c6f5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:11.388728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:11.412970 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:11.464849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750e4a7-7831-412c-82a6-dd001956b0b6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:11.475294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3c55e-dcf8-4941-b551-7db721c0c6f5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:11.512193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9281a-62fe-4ce5-a13a-00b0e6392921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:11.512888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9281a-62fe-4ce5-a13a-00b0e6392921 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.513181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9281a-62fe-4ce5-a13a-00b0e6392921 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.513467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9281a-62fe-4ce5-a13a-00b0e6392921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.514391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9281a-62fe-4ce5-a13a-00b0e6392921 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125, user.id=e2d188b242aa4be3ac73c99d08c98dbe) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:11.514712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9281a-62fe-4ce5-a13a-00b0e6392921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:11.516269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9281a-62fe-4ce5-a13a-00b0e6392921 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:11.522808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967c07b-a519-43dd-b910-16af8f8b1352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:11.523014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 84/161] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:11 2022] GET /identity/v3/role_assignments?user.id=e2d188b242aa4be3ac73c99d08c98dbe&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:11.523811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967c07b-a519-43dd-b910-16af8f8b1352 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.524161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9502f-0cdb-4006-97aa-8076d3bb9ff2 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.524282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967c07b-a519-43dd-b910-16af8f8b1352 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.524453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9502f-0cdb-4006-97aa-8076d3bb9ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.524548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967c07b-a519-43dd-b910-16af8f8b1352 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.524733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9502f-0cdb-4006-97aa-8076d3bb9ff2 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.525238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967c07b-a519-43dd-b910-16af8f8b1352 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:11.526091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967c07b-a519-43dd-b910-16af8f8b1352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:11.526794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 85/162] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:11.527725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1211e-f729-42a1-a75a-5ae3f9dde28c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.528016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1211e-f729-42a1-a75a-5ae3f9dde28c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.528297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1211e-f729-42a1-a75a-5ae3f9dde28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.528398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967c07b-a519-43dd-b910-16af8f8b1352 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:11.553752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 78/163] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:11.554687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f3899-5533-4870-85fc-154d1e7e430f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.555031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f3899-5533-4870-85fc-154d1e7e430f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.555313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f3899-5533-4870-85fc-154d1e7e430f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.561736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1de1211e-f729-42a1-a75a-5ae3f9dde28c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:11.561736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1de1211e-f729-42a1-a75a-5ae3f9dde28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:11.578832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f17f3899-5533-4870-85fc-154d1e7e430f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:11.580183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f17f3899-5533-4870-85fc-154d1e7e430f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:11.678059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 86/164] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:11.678957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1211e-f729-42a1-a75a-5ae3f9dde28c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:11.691687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 79/165] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:11.711047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f3899-5533-4870-85fc-154d1e7e430f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:11.765861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:11.766729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:11.830384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1211e-f729-42a1-a75a-5ae3f9dde28c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:11.864755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f3899-5533-4870-85fc-154d1e7e430f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:11.879744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50996a-663d-4e80-87e2-e24fa525dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:11.881977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50996a-663d-4e80-87e2-e24fa525dd8d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.881977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50996a-663d-4e80-87e2-e24fa525dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.881977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50996a-663d-4e80-87e2-e24fa525dd8d None admin] PATH_INFO: `/v3/users/0e961d4918e44e499e44d50410595f6d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.883517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50996a-663d-4e80-87e2-e24fa525dd8d None admin] RBAC: Authorizing `identity:get_user(user_id=0e961d4918e44e499e44d50410595f6d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:11.885621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50996a-663d-4e80-87e2-e24fa525dd8d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:11.887497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 87/166] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:11 2022] GET /identity/v3/users/0e961d4918e44e499e44d50410595f6d => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:11.888302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50996a-663d-4e80-87e2-e24fa525dd8d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:11.935431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e12e-309d-414b-b327-49bd4fcf5892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:11.936143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e12e-309d-414b-b327-49bd4fcf5892 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:11.936675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e12e-309d-414b-b327-49bd4fcf5892 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:11.937141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e12e-309d-414b-b327-49bd4fcf5892 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:11.964618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88e12e-309d-414b-b327-49bd4fcf5892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0b88e12e-309d-414b-b327-49bd4fcf5892 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:11.972661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:11.973667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:11.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:11.976133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 80/167] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:11.977119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e12e-309d-414b-b327-49bd4fcf5892 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:12.018612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50996a-663d-4e80-87e2-e24fa525dd8d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:12.047551 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:12.061294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd11f2c-8cf4-420b-8dfa-065080845ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:12.062509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd11f2c-8cf4-420b-8dfa-065080845ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:12.063071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd11f2c-8cf4-420b-8dfa-065080845ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:12.063553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd11f2c-8cf4-420b-8dfa-065080845ce5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:12.075865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd11f2c-8cf4-420b-8dfa-065080845ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-9cd11f2c-8cf4-420b-8dfa-065080845ce5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:12.084291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:12.085239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:12.086429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:12.089166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 88/168] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:12.090349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd11f2c-8cf4-420b-8dfa-065080845ce5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:12.122115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:12.142338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e12e-309d-414b-b327-49bd4fcf5892 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:12.211858 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd11f2c-8cf4-420b-8dfa-065080845ce5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:12.238049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03311ce5-d504-4f80-b4b0-c510ebedc4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:12.239552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03311ce5-d504-4f80-b4b0-c510ebedc4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:12.240048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03311ce5-d504-4f80-b4b0-c510ebedc4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:12.240531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03311ce5-d504-4f80-b4b0-c510ebedc4a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:12.277957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663eda1-b76d-46f6-9306-16b4688f0196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:12.279825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663eda1-b76d-46f6-9306-16b4688f0196 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:12.281395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663eda1-b76d-46f6-9306-16b4688f0196 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:12.281957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663eda1-b76d-46f6-9306-16b4688f0196 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:12.284191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663eda1-b76d-46f6-9306-16b4688f0196 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:12.286265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663eda1-b76d-46f6-9306-16b4688f0196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:12.289063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663eda1-b76d-46f6-9306-16b4688f0196 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:12.299676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 89/169] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:12.300868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663eda1-b76d-46f6-9306-16b4688f0196 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:12.385909 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03311ce5-d504-4f80-b4b0-c510ebedc4a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:12.388566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03311ce5-d504-4f80-b4b0-c510ebedc4a6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:12.454080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:12.498657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:12.506095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 81/170] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:12.512850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03311ce5-d504-4f80-b4b0-c510ebedc4a6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:12.567672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663eda1-b76d-46f6-9306-16b4688f0196 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:12.629039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03311ce5-d504-4f80-b4b0-c510ebedc4a6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:12.640501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ed4ab-d66c-45ca-9e80-2b79e2ddb54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:12.641196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed4ab-d66c-45ca-9e80-2b79e2ddb54a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:12.641482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed4ab-d66c-45ca-9e80-2b79e2ddb54a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:12.641803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed4ab-d66c-45ca-9e80-2b79e2ddb54a None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:12.642577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ed4ab-d66c-45ca-9e80-2b79e2ddb54a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:12.642889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ed4ab-d66c-45ca-9e80-2b79e2ddb54a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:12.644302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ed4ab-d66c-45ca-9e80-2b79e2ddb54a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:12.661925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 90/171] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:12.662807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ed4ab-d66c-45ca-9e80-2b79e2ddb54a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:12.668402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c06af-d0a1-4530-8c61-94fa4a0c1d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:12.669345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c06af-d0a1-4530-8c61-94fa4a0c1d71 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:12.669851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c06af-d0a1-4530-8c61-94fa4a0c1d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:12.670130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c06af-d0a1-4530-8c61-94fa4a0c1d71 None admin] PATH_INFO: `/v3/users/0e961d4918e44e499e44d50410595f6d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:12.671501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c06af-d0a1-4530-8c61-94fa4a0c1d71 None admin] RBAC: Authorizing `identity:get_user(user_id=0e961d4918e44e499e44d50410595f6d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:12.673208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c06af-d0a1-4530-8c61-94fa4a0c1d71 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:12.675335 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 82/172] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:12 2022] GET /identity/v3/users/0e961d4918e44e499e44d50410595f6d => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:12.676176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c06af-d0a1-4530-8c61-94fa4a0c1d71 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:12.790281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:12.874730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c06af-d0a1-4530-8c61-94fa4a0c1d71 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:12.889561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ed4ab-d66c-45ca-9e80-2b79e2ddb54a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:12.910109 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:12.954757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88369fe-f8ee-489f-8b52-ce44f338c2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:12.955674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88369fe-f8ee-489f-8b52-ce44f338c2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:12.956160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88369fe-f8ee-489f-8b52-ce44f338c2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:12.956512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88369fe-f8ee-489f-8b52-ce44f338c2c9 None admin] PATH_INFO: `/v3/users/506e319379084a9db24f03bbe0c0e2be` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:12.958083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88369fe-f8ee-489f-8b52-ce44f338c2c9 None admin] RBAC: Authorizing `identity:get_user(user_id=506e319379084a9db24f03bbe0c0e2be)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:12.959798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88369fe-f8ee-489f-8b52-ce44f338c2c9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:12.962341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 83/173] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:12 2022] GET /identity/v3/users/506e319379084a9db24f03bbe0c0e2be => generated 289 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:12.963917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88369fe-f8ee-489f-8b52-ce44f338c2c9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:12.979147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81fe96-7ddb-47a0-afec-fc910abcc5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:12.980056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81fe96-7ddb-47a0-afec-fc910abcc5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:12.980342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81fe96-7ddb-47a0-afec-fc910abcc5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:12.980621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81fe96-7ddb-47a0-afec-fc910abcc5b2 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:12.986524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81fe96-7ddb-47a0-afec-fc910abcc5b2 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:12.988028 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81fe96-7ddb-47a0-afec-fc910abcc5b2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:12.989841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 91/174] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:12 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:12.990735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81fe96-7ddb-47a0-afec-fc910abcc5b2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:13.084845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88369fe-f8ee-489f-8b52-ce44f338c2c9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:13.151727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:13.187398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2bf7d-fe0a-4cfa-9586-94d12598be86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:13.188355 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2bf7d-fe0a-4cfa-9586-94d12598be86 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:13.188708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2bf7d-fe0a-4cfa-9586-94d12598be86 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:13.189047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2bf7d-fe0a-4cfa-9586-94d12598be86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:13.199556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81fe96-7ddb-47a0-afec-fc910abcc5b2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:13.266563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ad29d-fd5d-4dfb-9c9d-f9f29ca74f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:13.268890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ad29d-fd5d-4dfb-9c9d-f9f29ca74f71 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:13.272895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ad29d-fd5d-4dfb-9c9d-f9f29ca74f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:13.273502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ad29d-fd5d-4dfb-9c9d-f9f29ca74f71 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:13.276539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ad29d-fd5d-4dfb-9c9d-f9f29ca74f71 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:13.280473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ad29d-fd5d-4dfb-9c9d-f9f29ca74f71 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:13.289578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 92/175] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:12 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:13.290852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ad29d-fd5d-4dfb-9c9d-f9f29ca74f71 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:13.370691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2bf7d-fe0a-4cfa-9586-94d12598be86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:13.372379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2bf7d-fe0a-4cfa-9586-94d12598be86 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:13.374546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:13.488106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ad29d-fd5d-4dfb-9c9d-f9f29ca74f71 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:13.556455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 84/176] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:13.557699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2bf7d-fe0a-4cfa-9586-94d12598be86 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:13.586701 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:13.593929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa627-607f-4881-9a33-b9b493ad0dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:13.594648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa627-607f-4881-9a33-b9b493ad0dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:13.594890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa627-607f-4881-9a33-b9b493ad0dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:13.595168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa627-607f-4881-9a33-b9b493ad0dd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:13.596054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa627-607f-4881-9a33-b9b493ad0dd5 None admin] RBAC: Adding query filter params (role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125, user.id=0e961d4918e44e499e44d50410595f6d) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:13.596528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa627-607f-4881-9a33-b9b493ad0dd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:13.598382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa627-607f-4881-9a33-b9b493ad0dd5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:13.610481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 93/177] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:13 2022] GET /identity/v3/role_assignments?user.id=0e961d4918e44e499e44d50410595f6d&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 608 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:13.611187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa627-607f-4881-9a33-b9b493ad0dd5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:13.738392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2bf7d-fe0a-4cfa-9586-94d12598be86 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:13.800057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85791a-042b-474e-aed1-17117a0e93c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:13.801139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85791a-042b-474e-aed1-17117a0e93c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:13.802092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85791a-042b-474e-aed1-17117a0e93c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:13.802497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85791a-042b-474e-aed1-17117a0e93c7 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/users/506e319379084a9db24f03bbe0c0e2be/roles/443ba7d155f54b0f85e995b760bd77bb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:13.806224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85791a-042b-474e-aed1-17117a0e93c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=574e2d880a9446af80a6bbdfa7f0b125, user_id=506e319379084a9db24f03bbe0c0e2be, role_id=443ba7d155f54b0f85e995b760bd77bb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:13.808398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85791a-042b-474e-aed1-17117a0e93c7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:13.826759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 85/178] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:07:13 2022] PUT /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/users/506e319379084a9db24f03bbe0c0e2be/roles/443ba7d155f54b0f85e995b760bd77bb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:13.839355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa627-607f-4881-9a33-b9b493ad0dd5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:13.860468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:13.907783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003e6d8-21b5-4f19-968b-4cec4ab53d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:13.908719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e6d8-21b5-4f19-968b-4cec4ab53d0e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:13.909009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e6d8-21b5-4f19-968b-4cec4ab53d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:13.909289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e6d8-21b5-4f19-968b-4cec4ab53d0e None admin] PATH_INFO: `/v3/projects/8239487747da496d8247459b6e4c7051` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:13.910929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003e6d8-21b5-4f19-968b-4cec4ab53d0e None admin] RBAC: Authorizing `identity:get_project(project_id=8239487747da496d8247459b6e4c7051)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:13.912598 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003e6d8-21b5-4f19-968b-4cec4ab53d0e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:13.914964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 94/179] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:13 2022] GET /identity/v3/projects/8239487747da496d8247459b6e4c7051 => generated 312 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:13.922135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85791a-042b-474e-aed1-17117a0e93c7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:14.044467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:14.073382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85791a-042b-474e-aed1-17117a0e93c7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:14.124374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d7a41-f2ee-47c8-b95a-3c2b2bef12d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:14.125058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7a41-f2ee-47c8-b95a-3c2b2bef12d8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:14.125340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7a41-f2ee-47c8-b95a-3c2b2bef12d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:14.125618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7a41-f2ee-47c8-b95a-3c2b2bef12d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:14.128020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d7a41-f2ee-47c8-b95a-3c2b2bef12d8 None admin] RBAC: Adding query filter params (role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=8239487747da496d8247459b6e4c7051, user.id=0e961d4918e44e499e44d50410595f6d) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:14.128020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d7a41-f2ee-47c8-b95a-3c2b2bef12d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:14.128357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d7a41-f2ee-47c8-b95a-3c2b2bef12d8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:14.137607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 86/180] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:13 2022] GET /identity/v3/role_assignments?user.id=0e961d4918e44e499e44d50410595f6d&scope.project.id=8239487747da496d8247459b6e4c7051&role.id=4a06d59562de4d598671a462e3eaf62a => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:14.622905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb67a0-319f-415b-b463-ce3062f46b79 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:14.623536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb67a0-319f-415b-b463-ce3062f46b79 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:14.624010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb67a0-319f-415b-b463-ce3062f46b79 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:14.625618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 95/181] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:14.631196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e388d-0004-4f76-b4d0-b75a77eeab8f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:14.631677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e388d-0004-4f76-b4d0-b75a77eeab8f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:14.631754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e388d-0004-4f76-b4d0-b75a77eeab8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:14.652869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2c1e388d-0004-4f76-b4d0-b75a77eeab8f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:14.655195 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2c1e388d-0004-4f76-b4d0-b75a77eeab8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:14.739701 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 87/182] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:14.766600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb67a0-319f-415b-b463-ce3062f46b79 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:14.772358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdb166-15b5-4b74-9477-2430c3183288 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:14.772650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdb166-15b5-4b74-9477-2430c3183288 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:14.772937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdb166-15b5-4b74-9477-2430c3183288 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:14.774514 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 88/183] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:14.780691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5002366-d6b9-4593-85ee-3f5493d13a7c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:14.780988 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5002366-d6b9-4593-85ee-3f5493d13a7c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:14.781271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5002366-d6b9-4593-85ee-3f5493d13a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:14.810952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e5002366-d6b9-4593-85ee-3f5493d13a7c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:14.812378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e5002366-d6b9-4593-85ee-3f5493d13a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:14.900135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 89/184] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:14.902584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f24192-1e46-4923-b5d0-d18cc1b9a9db None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:14.903043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f24192-1e46-4923-b5d0-d18cc1b9a9db None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:14.903328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f24192-1e46-4923-b5d0-d18cc1b9a9db None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:14.905377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 90/185] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:14.910489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb67a0-319f-415b-b463-ce3062f46b79 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:14.915218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0c7b-a65e-40d4-8d8d-98f6fb4fedb2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:14.915507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0c7b-a65e-40d4-8d8d-98f6fb4fedb2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:14.915786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0c7b-a65e-40d4-8d8d-98f6fb4fedb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:14.940908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-089d0c7b-a65e-40d4-8d8d-98f6fb4fedb2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:14.943092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-089d0c7b-a65e-40d4-8d8d-98f6fb4fedb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:14.946250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:14.954020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b498fc-a9b5-4d57-a422-5f3086129c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:14.954824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b498fc-a9b5-4d57-a422-5f3086129c16 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:14.955111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b498fc-a9b5-4d57-a422-5f3086129c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:14.955384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b498fc-a9b5-4d57-a422-5f3086129c16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:14.956558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b498fc-a9b5-4d57-a422-5f3086129c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:14.958073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b498fc-a9b5-4d57-a422-5f3086129c16 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:14.959706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 96/186] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:07:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:14.960487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b498fc-a9b5-4d57-a422-5f3086129c16 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:15.011782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:15.021745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 91/187] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:15.022581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d0c7b-a65e-40d4-8d8d-98f6fb4fedb2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:15.075617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b498fc-a9b5-4d57-a422-5f3086129c16 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:15.122407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8821fa-516a-4e27-a38a-79648ab8ef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:15.123466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8821fa-516a-4e27-a38a-79648ab8ef7a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:15.123754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8821fa-516a-4e27-a38a-79648ab8ef7a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:15.124028 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8821fa-516a-4e27-a38a-79648ab8ef7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:15.137300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8821fa-516a-4e27-a38a-79648ab8ef7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:15.140581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d0c7b-a65e-40d4-8d8d-98f6fb4fedb2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1a8821fa-516a-4e27-a38a-79648ab8ef7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:15.144952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:15.146099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:15.146919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 97/188] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:07:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:15.147575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8821fa-516a-4e27-a38a-79648ab8ef7a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:15.182223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd61cbb-c52e-4230-9fa1-4418066fd97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:15.182918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd61cbb-c52e-4230-9fa1-4418066fd97d None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:15.183203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd61cbb-c52e-4230-9fa1-4418066fd97d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:15.183490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd61cbb-c52e-4230-9fa1-4418066fd97d None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:15.184200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd61cbb-c52e-4230-9fa1-4418066fd97d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:15.185666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd61cbb-c52e-4230-9fa1-4418066fd97d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:15.195075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:15.209880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 92/189] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:07:15 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:07:15.222821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd61cbb-c52e-4230-9fa1-4418066fd97d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:15.274173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8821fa-516a-4e27-a38a-79648ab8ef7a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:15.278841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:15.332324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09ee6c-1073-46fe-ac54-ca5d13634143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:15.333073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09ee6c-1073-46fe-ac54-ca5d13634143 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:15.333363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09ee6c-1073-46fe-ac54-ca5d13634143 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:15.333637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09ee6c-1073-46fe-ac54-ca5d13634143 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:15.352245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09ee6c-1073-46fe-ac54-ca5d13634143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-5e09ee6c-1073-46fe-ac54-ca5d13634143 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:15.364942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:15.366043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:15.366861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:15.369995 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 98/190] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:15.376118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09ee6c-1073-46fe-ac54-ca5d13634143 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:15.388684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd61cbb-c52e-4230-9fa1-4418066fd97d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:15.435978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5c6ce-6a2c-41f1-8a3e-d49dab379b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:15.436651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5c6ce-6a2c-41f1-8a3e-d49dab379b12 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:15.436935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5c6ce-6a2c-41f1-8a3e-d49dab379b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:15.437208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5c6ce-6a2c-41f1-8a3e-d49dab379b12 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:15.438024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5c6ce-6a2c-41f1-8a3e-d49dab379b12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:15.438397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5c6ce-6a2c-41f1-8a3e-d49dab379b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:15.439879 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5c6ce-6a2c-41f1-8a3e-d49dab379b12 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:15.450583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 93/191] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:15.456021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5c6ce-6a2c-41f1-8a3e-d49dab379b12 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:15.504788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09ee6c-1073-46fe-ac54-ca5d13634143 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:15.535584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:15.549622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661eb31-6a4c-427a-b84e-403967c968ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:15.550382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661eb31-6a4c-427a-b84e-403967c968ac None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:15.550947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661eb31-6a4c-427a-b84e-403967c968ac None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:15.551674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661eb31-6a4c-427a-b84e-403967c968ac None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:15.552563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661eb31-6a4c-427a-b84e-403967c968ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:15.552876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661eb31-6a4c-427a-b84e-403967c968ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:15.556011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661eb31-6a4c-427a-b84e-403967c968ac None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:15.565888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 99/192] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:15.573821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661eb31-6a4c-427a-b84e-403967c968ac None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:15.608569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5c6ce-6a2c-41f1-8a3e-d49dab379b12 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:15.611630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:15.678847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25954e04-b48e-402f-8964-abdcc66ee499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:15.679749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954e04-b48e-402f-8964-abdcc66ee499 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:15.680034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954e04-b48e-402f-8964-abdcc66ee499 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:15.680312 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954e04-b48e-402f-8964-abdcc66ee499 None admin] PATH_INFO: `/v3/users/99b2b46dd8ef4441ad28e83a8205b76d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:15.723413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25954e04-b48e-402f-8964-abdcc66ee499 None admin] RBAC: Authorizing `identity:get_user(user_id=99b2b46dd8ef4441ad28e83a8205b76d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:15.724833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25954e04-b48e-402f-8964-abdcc66ee499 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:15.726071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661eb31-6a4c-427a-b84e-403967c968ac None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:15.726628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 94/193] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:15 2022] GET /identity/v3/users/99b2b46dd8ef4441ad28e83a8205b76d => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:15.751257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25954e04-b48e-402f-8964-abdcc66ee499 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:15.776104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062382c4-4b61-42c4-8f39-1960bbabf0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:15.778289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062382c4-4b61-42c4-8f39-1960bbabf0bf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:15.779472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062382c4-4b61-42c4-8f39-1960bbabf0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:15.780951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062382c4-4b61-42c4-8f39-1960bbabf0bf None admin] PATH_INFO: `/v3/users/99b2b46dd8ef4441ad28e83a8205b76d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:15.782545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062382c4-4b61-42c4-8f39-1960bbabf0bf None admin] RBAC: Authorizing `identity:get_user(user_id=99b2b46dd8ef4441ad28e83a8205b76d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:15.785276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062382c4-4b61-42c4-8f39-1960bbabf0bf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:15.789741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 100/194] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:15 2022] GET /identity/v3/users/99b2b46dd8ef4441ad28e83a8205b76d => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:15.795542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062382c4-4b61-42c4-8f39-1960bbabf0bf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:15.909670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25954e04-b48e-402f-8964-abdcc66ee499 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:15.926188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:15.955580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abd254-64e2-493a-9308-5199027c9494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:15.956663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abd254-64e2-493a-9308-5199027c9494 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:15.958295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abd254-64e2-493a-9308-5199027c9494 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:15.958796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abd254-64e2-493a-9308-5199027c9494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:16.013556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:16.032224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062382c4-4b61-42c4-8f39-1960bbabf0bf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:16.104356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ed21-c4b2-4b00-82ce-06d4253b177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:16.105277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ed21-c4b2-4b00-82ce-06d4253b177d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:16.105771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ed21-c4b2-4b00-82ce-06d4253b177d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:16.106192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ed21-c4b2-4b00-82ce-06d4253b177d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:16.108553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abd254-64e2-493a-9308-5199027c9494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:16.111917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abd254-64e2-493a-9308-5199027c9494 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:16.247646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87ed21-c4b2-4b00-82ce-06d4253b177d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:16.249586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87ed21-c4b2-4b00-82ce-06d4253b177d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:16.308545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 95/195] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:16.314307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abd254-64e2-493a-9308-5199027c9494 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:16.345892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:16.427238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:16.436764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 101/196] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:16.445924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ed21-c4b2-4b00-82ce-06d4253b177d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:16.554695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abd254-64e2-493a-9308-5199027c9494 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:16.638435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d923-5260-4e04-a293-96acd7964e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:16.639315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d923-5260-4e04-a293-96acd7964e67 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:16.639603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d923-5260-4e04-a293-96acd7964e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:16.639886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d923-5260-4e04-a293-96acd7964e67 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:16.642237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d923-5260-4e04-a293-96acd7964e67 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:16.643815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d923-5260-4e04-a293-96acd7964e67 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:16.649475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 96/197] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:16 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:16.651932 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d923-5260-4e04-a293-96acd7964e67 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:16.660139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ed21-c4b2-4b00-82ce-06d4253b177d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:16.733942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f9736-22a5-41a7-9d4d-d14a8206b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:16.734829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f9736-22a5-41a7-9d4d-d14a8206b489 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:16.735114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f9736-22a5-41a7-9d4d-d14a8206b489 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:16.735387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f9736-22a5-41a7-9d4d-d14a8206b489 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:16.736817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f9736-22a5-41a7-9d4d-d14a8206b489 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:16.738406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f9736-22a5-41a7-9d4d-d14a8206b489 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:16.740597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 102/198] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:16 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:16.747224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f9736-22a5-41a7-9d4d-d14a8206b489 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:16.770111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:16.802324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d923-5260-4e04-a293-96acd7964e67 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:16.843502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e51f23-a243-46ae-be6b-3ed9c625ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:16.844172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e51f23-a243-46ae-be6b-3ed9c625ca2f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:16.844456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e51f23-a243-46ae-be6b-3ed9c625ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:16.844735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e51f23-a243-46ae-be6b-3ed9c625ca2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:16.845688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e51f23-a243-46ae-be6b-3ed9c625ca2f None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=908f423535c84512acd6b17bd7b971ce, user.id=99b2b46dd8ef4441ad28e83a8205b76d) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:16.846134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e51f23-a243-46ae-be6b-3ed9c625ca2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:16.847613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e51f23-a243-46ae-be6b-3ed9c625ca2f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:16.848434 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:16.856933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 97/199] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:16 2022] GET /identity/v3/role_assignments?user.id=99b2b46dd8ef4441ad28e83a8205b76d&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:16.866763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8017a0-062c-43f4-880e-e15153d1b2f4 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:16.867048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8017a0-062c-43f4-880e-e15153d1b2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:16.867318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8017a0-062c-43f4-880e-e15153d1b2f4 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:16.868784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 98/200] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:16.887490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1d58-0b61-41a3-9a79-4dba72211bf5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:16.887830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1d58-0b61-41a3-9a79-4dba72211bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:16.888046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d1d58-0b61-41a3-9a79-4dba72211bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:16.903068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f9736-22a5-41a7-9d4d-d14a8206b489 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:16.929808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a06d1d58-0b61-41a3-9a79-4dba72211bf5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:16.931005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a06d1d58-0b61-41a3-9a79-4dba72211bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:16.943810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f035bd-91c0-4b8f-a9b2-70199d3c8a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:16.944482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f035bd-91c0-4b8f-a9b2-70199d3c8a13 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:16.944764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f035bd-91c0-4b8f-a9b2-70199d3c8a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:16.945040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f035bd-91c0-4b8f-a9b2-70199d3c8a13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:16.946564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f035bd-91c0-4b8f-a9b2-70199d3c8a13 None admin] RBAC: Adding query filter params (role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=908f423535c84512acd6b17bd7b971ce, user.id=99b2b46dd8ef4441ad28e83a8205b76d) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:16.946891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f035bd-91c0-4b8f-a9b2-70199d3c8a13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:16.949779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f035bd-91c0-4b8f-a9b2-70199d3c8a13 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:16.957466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 103/201] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:16 2022] GET /identity/v3/role_assignments?user.id=99b2b46dd8ef4441ad28e83a8205b76d&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:17.055559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 99/202] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:17.096098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f035bd-91c0-4b8f-a9b2-70199d3c8a13 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:17.228221 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f035bd-91c0-4b8f-a9b2-70199d3c8a13 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:17.258752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:17.268750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879e3fb-3cd1-4610-a90a-b01e0b0a710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:17.269562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879e3fb-3cd1-4610-a90a-b01e0b0a710e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:17.269875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879e3fb-3cd1-4610-a90a-b01e0b0a710e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:17.270144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879e3fb-3cd1-4610-a90a-b01e0b0a710e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:17.283223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879e3fb-3cd1-4610-a90a-b01e0b0a710e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f879e3fb-3cd1-4610-a90a-b01e0b0a710e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:17.291093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:17.292023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:17.292806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:17.293436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 104/203] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:17.302869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d1d58-0b61-41a3-9a79-4dba72211bf5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:17.400204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:17.438220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d1d58-0b61-41a3-9a79-4dba72211bf5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:17.481543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a323a26-c80b-4680-9d7e-0b7a7fd5889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:17.482515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a323a26-c80b-4680-9d7e-0b7a7fd5889f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:17.482806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a323a26-c80b-4680-9d7e-0b7a7fd5889f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:17.483087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a323a26-c80b-4680-9d7e-0b7a7fd5889f None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:17.483860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a323a26-c80b-4680-9d7e-0b7a7fd5889f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:17.484168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a323a26-c80b-4680-9d7e-0b7a7fd5889f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:17.485600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a323a26-c80b-4680-9d7e-0b7a7fd5889f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:17.496503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 100/204] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:17.502100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879e3fb-3cd1-4610-a90a-b01e0b0a710e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:17.663203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879e3fb-3cd1-4610-a90a-b01e0b0a710e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:17.711556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be0987-b0ca-4b2b-8b56-4db941cd82a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:17.712467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be0987-b0ca-4b2b-8b56-4db941cd82a3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:17.712780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be0987-b0ca-4b2b-8b56-4db941cd82a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:17.713131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be0987-b0ca-4b2b-8b56-4db941cd82a3 None admin] PATH_INFO: `/v3/users/506e319379084a9db24f03bbe0c0e2be` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:17.717546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be0987-b0ca-4b2b-8b56-4db941cd82a3 None admin] RBAC: Authorizing `identity:get_user(user_id=506e319379084a9db24f03bbe0c0e2be)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:17.719130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be0987-b0ca-4b2b-8b56-4db941cd82a3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:17.720979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 105/205] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:17 2022] GET /identity/v3/users/506e319379084a9db24f03bbe0c0e2be => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:17.727564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a323a26-c80b-4680-9d7e-0b7a7fd5889f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:17.874649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a323a26-c80b-4680-9d7e-0b7a7fd5889f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:17.928159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c39f4e-a4fb-4029-a9e3-1a14fe423564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:17.928735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c39f4e-a4fb-4029-a9e3-1a14fe423564 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:17.929018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c39f4e-a4fb-4029-a9e3-1a14fe423564 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:17.929294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c39f4e-a4fb-4029-a9e3-1a14fe423564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:17.939370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:18.044343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c39f4e-a4fb-4029-a9e3-1a14fe423564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:18.046953 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c39f4e-a4fb-4029-a9e3-1a14fe423564 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:18.123787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ae14-91db-4711-b597-adb128b4fe9e None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:18.124757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ae14-91db-4711-b597-adb128b4fe9e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:18.124757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ae14-91db-4711-b597-adb128b4fe9e None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:18.125746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 106/206] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:18.131402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb8d2d-b980-4912-8900-ecbe2e411a91 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:18.131743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb8d2d-b980-4912-8900-ecbe2e411a91 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:18.131971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb8d2d-b980-4912-8900-ecbe2e411a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:18.167443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-68bb8d2d-b980-4912-8900-ecbe2e411a91 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:18.168743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-68bb8d2d-b980-4912-8900-ecbe2e411a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:18.250887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 101/207] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:18.252445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c39f4e-a4fb-4029-a9e3-1a14fe423564 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:18.285698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:18.320074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 107/208] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:18.328598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:18.366996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb8d2d-b980-4912-8900-ecbe2e411a91 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:18.437345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c39f4e-a4fb-4029-a9e3-1a14fe423564 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:18.525048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b1ed7-7210-4f74-bffb-9d83a702e2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:18.527272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b1ed7-7210-4f74-bffb-9d83a702e2dc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:18.528168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b1ed7-7210-4f74-bffb-9d83a702e2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:18.528931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b1ed7-7210-4f74-bffb-9d83a702e2dc None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:18.531566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b1ed7-7210-4f74-bffb-9d83a702e2dc None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:18.534872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b1ed7-7210-4f74-bffb-9d83a702e2dc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:18.538976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 102/209] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:18 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:18.543427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aac159-debc-43cc-88e7-fb1403ec1d78 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:18.544238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aac159-debc-43cc-88e7-fb1403ec1d78 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:18.544970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aac159-debc-43cc-88e7-fb1403ec1d78 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:18.548439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 103/210] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:18.550576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aac159-debc-43cc-88e7-fb1403ec1d78 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:18.551567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb8d2d-b980-4912-8900-ecbe2e411a91 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:18.596994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa1fd2-15ae-43cd-9153-a8e4b96e2814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:18.597876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa1fd2-15ae-43cd-9153-a8e4b96e2814 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:18.598161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa1fd2-15ae-43cd-9153-a8e4b96e2814 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:18.598421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa1fd2-15ae-43cd-9153-a8e4b96e2814 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:18.599615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa1fd2-15ae-43cd-9153-a8e4b96e2814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:18.601757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa1fd2-15ae-43cd-9153-a8e4b96e2814 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:18.604144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 108/211] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:07:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:18.606393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4abf84-8cee-46de-8115-30968a67c174 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:18.606761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4abf84-8cee-46de-8115-30968a67c174 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:18.606984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4abf84-8cee-46de-8115-30968a67c174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:18.628826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4b4abf84-8cee-46de-8115-30968a67c174 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:18.630376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4b4abf84-8cee-46de-8115-30968a67c174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:18.705456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:18.741235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aac159-debc-43cc-88e7-fb1403ec1d78 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:18.747552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 109/212] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:18.749396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4abf84-8cee-46de-8115-30968a67c174 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:18.757478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:18.809895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7619872-ed5e-498a-a73f-9c00a54dc9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:18.810486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7619872-ed5e-498a-a73f-9c00a54dc9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:18.810772 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7619872-ed5e-498a-a73f-9c00a54dc9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:18.811029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7619872-ed5e-498a-a73f-9c00a54dc9e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:18.811867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7619872-ed5e-498a-a73f-9c00a54dc9e1 None admin] RBAC: Adding query filter params (role.id=443ba7d155f54b0f85e995b760bd77bb, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125, user.id=506e319379084a9db24f03bbe0c0e2be) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:18.812177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7619872-ed5e-498a-a73f-9c00a54dc9e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:18.813741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7619872-ed5e-498a-a73f-9c00a54dc9e1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:18.829921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 104/213] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:18 2022] GET /identity/v3/role_assignments?user.id=506e319379084a9db24f03bbe0c0e2be&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=443ba7d155f54b0f85e995b760bd77bb => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:18.831079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7619872-ed5e-498a-a73f-9c00a54dc9e1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:18.958930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7619872-ed5e-498a-a73f-9c00a54dc9e1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:19.002118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c227b-1b57-4afe-969e-19eef856dd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:19.002962 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c227b-1b57-4afe-969e-19eef856dd56 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:19.003250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c227b-1b57-4afe-969e-19eef856dd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:19.003532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c227b-1b57-4afe-969e-19eef856dd56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:19.016811 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4abf84-8cee-46de-8115-30968a67c174 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:19.017068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c227b-1b57-4afe-969e-19eef856dd56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9a7c227b-1b57-4afe-969e-19eef856dd56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:19.026274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:19.027881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:19.029351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:19.032446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 105/214] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:19.033897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c227b-1b57-4afe-969e-19eef856dd56 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:19.070376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:19.098601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61651d2-3641-4819-8e2d-94526a6f7776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:19.099317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61651d2-3641-4819-8e2d-94526a6f7776 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:19.099606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61651d2-3641-4819-8e2d-94526a6f7776 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:19.099885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61651d2-3641-4819-8e2d-94526a6f7776 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:19.100625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61651d2-3641-4819-8e2d-94526a6f7776 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:19.102180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61651d2-3641-4819-8e2d-94526a6f7776 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:19.156181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c227b-1b57-4afe-969e-19eef856dd56 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:19.156586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 110/215] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:07:18 2022] POST /identity/v3/users => generated 297 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:07:19.200887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574197f-b619-445b-96d5-c87532e0c783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:19.201643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574197f-b619-445b-96d5-c87532e0c783 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:19.201978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574197f-b619-445b-96d5-c87532e0c783 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:19.202235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574197f-b619-445b-96d5-c87532e0c783 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:19.203031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574197f-b619-445b-96d5-c87532e0c783 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:19.203340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574197f-b619-445b-96d5-c87532e0c783 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:19.204757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574197f-b619-445b-96d5-c87532e0c783 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:19.218741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 106/216] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:19.222871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61651d2-3641-4819-8e2d-94526a6f7776 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:19.281830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:19.366469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61651d2-3641-4819-8e2d-94526a6f7776 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:19.412799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2116c7-8105-44cd-a23d-c82d0776db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:19.413740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2116c7-8105-44cd-a23d-c82d0776db99 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:19.414031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2116c7-8105-44cd-a23d-c82d0776db99 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:19.414313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2116c7-8105-44cd-a23d-c82d0776db99 None admin] PATH_INFO: `/v3/users/bea64f92a6364520b93be53ce08ff4a9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:19.434762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2116c7-8105-44cd-a23d-c82d0776db99 None admin] RBAC: Authorizing `identity:get_user(user_id=bea64f92a6364520b93be53ce08ff4a9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:19.435268 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2116c7-8105-44cd-a23d-c82d0776db99 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:19.437340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 111/217] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:19 2022] GET /identity/v3/users/bea64f92a6364520b93be53ce08ff4a9 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:19.442764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574197f-b619-445b-96d5-c87532e0c783 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:19.565692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:19.578193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574197f-b619-445b-96d5-c87532e0c783 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:19.618916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502dbe4-48aa-4847-9aa1-ccd23959ecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:19.620108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502dbe4-48aa-4847-9aa1-ccd23959ecc4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:19.620396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502dbe4-48aa-4847-9aa1-ccd23959ecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:19.620668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502dbe4-48aa-4847-9aa1-ccd23959ecc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:19.704229 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502dbe4-48aa-4847-9aa1-ccd23959ecc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:19.705732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502dbe4-48aa-4847-9aa1-ccd23959ecc4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:19.822002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:19.836945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 107/218] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:19.840295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2116c7-8105-44cd-a23d-c82d0776db99 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:19.971936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:19.974696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2c294-7c9a-40ca-baee-7087518b4813 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:19.975148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2c294-7c9a-40ca-baee-7087518b4813 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:19.975303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2c294-7c9a-40ca-baee-7087518b4813 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:19.981060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 108/219] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:19.982050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2116c7-8105-44cd-a23d-c82d0776db99 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:19.998402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb98d9-4c03-44b4-b9e6-a72c256f950b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:19.998971 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb98d9-4c03-44b4-b9e6-a72c256f950b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:19.999425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb98d9-4c03-44b4-b9e6-a72c256f950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:20.030067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-76cb98d9-4c03-44b4-b9e6-a72c256f950b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:20.032604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-76cb98d9-4c03-44b4-b9e6-a72c256f950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:20.062520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110560a0-15dc-4b6c-acd9-9dc7f5adb51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:20.063433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110560a0-15dc-4b6c-acd9-9dc7f5adb51e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:20.063721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110560a0-15dc-4b6c-acd9-9dc7f5adb51e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:20.063996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110560a0-15dc-4b6c-acd9-9dc7f5adb51e None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:20.065341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110560a0-15dc-4b6c-acd9-9dc7f5adb51e None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:20.066909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110560a0-15dc-4b6c-acd9-9dc7f5adb51e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:20.068822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 112/220] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:19 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:20.075419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110560a0-15dc-4b6c-acd9-9dc7f5adb51e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:20.129047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 109/221] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:20.145043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d38cc-4b95-45ce-bafb-893f0a0e61db None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:20.145611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d38cc-4b95-45ce-bafb-893f0a0e61db None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:20.146116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d38cc-4b95-45ce-bafb-893f0a0e61db None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:20.147738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 110/222] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:20.155907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6f174-dc4d-484d-b5ab-4f9739509ee4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:20.156566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6f174-dc4d-484d-b5ab-4f9739509ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:20.157045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6f174-dc4d-484d-b5ab-4f9739509ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:20.176505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3bc6f174-dc4d-484d-b5ab-4f9739509ee4 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:20.177927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3bc6f174-dc4d-484d-b5ab-4f9739509ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:20.239558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110560a0-15dc-4b6c-acd9-9dc7f5adb51e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:20.261845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 111/223] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:20.262931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6f174-dc4d-484d-b5ab-4f9739509ee4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:20.291648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f73f67-76d7-429e-967e-7acf6b20225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:20.291648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f73f67-76d7-429e-967e-7acf6b20225b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:20.291648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f73f67-76d7-429e-967e-7acf6b20225b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:20.291648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f73f67-76d7-429e-967e-7acf6b20225b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:20.295453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f73f67-76d7-429e-967e-7acf6b20225b None admin] RBAC: Adding query filter params (role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=908f423535c84512acd6b17bd7b971ce, user.id=bea64f92a6364520b93be53ce08ff4a9) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:20.295776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f73f67-76d7-429e-967e-7acf6b20225b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:20.298110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f73f67-76d7-429e-967e-7acf6b20225b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:20.313086 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 113/224] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:20 2022] GET /identity/v3/role_assignments?user.id=bea64f92a6364520b93be53ce08ff4a9&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=4a06d59562de4d598671a462e3eaf62a => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:20.314597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f73f67-76d7-429e-967e-7acf6b20225b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:20.389979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:20.414165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:20.578895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f73f67-76d7-429e-967e-7acf6b20225b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:20.579476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6f174-dc4d-484d-b5ab-4f9739509ee4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:20.628612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411ea7b-28d1-4d20-a263-9fb965f745a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:20.629436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411ea7b-28d1-4d20-a263-9fb965f745a2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:20.629888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411ea7b-28d1-4d20-a263-9fb965f745a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:20.630301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411ea7b-28d1-4d20-a263-9fb965f745a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:20.646118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411ea7b-28d1-4d20-a263-9fb965f745a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-3411ea7b-28d1-4d20-a263-9fb965f745a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:20.654607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:20.655559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:20.656607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 114/225] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:20.672401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a09d6-ed60-4911-938d-3e12c89bee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:20.674574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a09d6-ed60-4911-938d-3e12c89bee79 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:20.675126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a09d6-ed60-4911-938d-3e12c89bee79 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:20.675707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a09d6-ed60-4911-938d-3e12c89bee79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:20.685145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411ea7b-28d1-4d20-a263-9fb965f745a2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:20.701117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a09d6-ed60-4911-938d-3e12c89bee79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8f2a09d6-ed60-4911-938d-3e12c89bee79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:20.723496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:20.725658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:20.727904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 112/226] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:07:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:20.736411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a09d6-ed60-4911-938d-3e12c89bee79 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:20.916396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:20.939679 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411ea7b-28d1-4d20-a263-9fb965f745a2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:20.974375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a09d6-ed60-4911-938d-3e12c89bee79 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:21.003801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:21.024379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f207-c295-471e-ba0a-7fc59be2711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:21.025883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f207-c295-471e-ba0a-7fc59be2711a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:21.028469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f207-c295-471e-ba0a-7fc59be2711a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:21.028986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f207-c295-471e-ba0a-7fc59be2711a None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:21.029981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f207-c295-471e-ba0a-7fc59be2711a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:21.030436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f207-c295-471e-ba0a-7fc59be2711a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:21.033469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f207-c295-471e-ba0a-7fc59be2711a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:21.053021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 115/227] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:21.057328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf6cea-8373-4444-9965-bff8396b9d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:21.058599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf6cea-8373-4444-9965-bff8396b9d8c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:21.059072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf6cea-8373-4444-9965-bff8396b9d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:21.059462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f207-c295-471e-ba0a-7fc59be2711a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:21.059705 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf6cea-8373-4444-9965-bff8396b9d8c None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:21.060663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf6cea-8373-4444-9965-bff8396b9d8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:21.061143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf6cea-8373-4444-9965-bff8396b9d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:21.062747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf6cea-8373-4444-9965-bff8396b9d8c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:21.073262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 113/228] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:21.082753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf6cea-8373-4444-9965-bff8396b9d8c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:21.268106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf6cea-8373-4444-9965-bff8396b9d8c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:21.271731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f207-c295-471e-ba0a-7fc59be2711a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:21.352905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28b24c-1506-4aab-9cb5-1b011d98fb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:21.355059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28b24c-1506-4aab-9cb5-1b011d98fb05 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:21.355565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28b24c-1506-4aab-9cb5-1b011d98fb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:21.356065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28b24c-1506-4aab-9cb5-1b011d98fb05 None admin] PATH_INFO: `/v3/users/99b2b46dd8ef4441ad28e83a8205b76d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:21.362290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28b24c-1506-4aab-9cb5-1b011d98fb05 None admin] RBAC: Authorizing `identity:get_user(user_id=99b2b46dd8ef4441ad28e83a8205b76d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:21.365139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28b24c-1506-4aab-9cb5-1b011d98fb05 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:21.368446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 114/229] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:21 2022] GET /identity/v3/users/99b2b46dd8ef4441ad28e83a8205b76d => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:21.377511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a8780-4048-4c64-b651-bc0dfc661a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:21.379109 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a8780-4048-4c64-b651-bc0dfc661a63 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:21.379607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a8780-4048-4c64-b651-bc0dfc661a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:21.380104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a8780-4048-4c64-b651-bc0dfc661a63 None admin] PATH_INFO: `/v3/users/99b2b46dd8ef4441ad28e83a8205b76d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:21.382768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28b24c-1506-4aab-9cb5-1b011d98fb05 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:21.385886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a8780-4048-4c64-b651-bc0dfc661a63 None admin] RBAC: Authorizing `identity:get_user(user_id=99b2b46dd8ef4441ad28e83a8205b76d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:21.394135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a8780-4048-4c64-b651-bc0dfc661a63 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:21.405211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 116/230] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:21 2022] GET /identity/v3/users/99b2b46dd8ef4441ad28e83a8205b76d => generated 283 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:21.415523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a8780-4048-4c64-b651-bc0dfc661a63 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:21.453175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:21.478257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:21.549238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28b24c-1506-4aab-9cb5-1b011d98fb05 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:21.652237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d436e-820a-4b24-9593-250a425dda35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:21.653808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d436e-820a-4b24-9593-250a425dda35 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:21.654295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d436e-820a-4b24-9593-250a425dda35 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:21.654789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d436e-820a-4b24-9593-250a425dda35 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:21.661072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d436e-820a-4b24-9593-250a425dda35 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:21.663865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d436e-820a-4b24-9593-250a425dda35 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:21.674373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 115/231] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:21 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:21.693667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d436e-820a-4b24-9593-250a425dda35 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:21.713733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a8780-4048-4c64-b651-bc0dfc661a63 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:21.811739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f98bbb-8471-4c60-bc07-d9ede58cd083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:21.812966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f98bbb-8471-4c60-bc07-d9ede58cd083 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:21.815298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f98bbb-8471-4c60-bc07-d9ede58cd083 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:21.815802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f98bbb-8471-4c60-bc07-d9ede58cd083 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:21.818691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f98bbb-8471-4c60-bc07-d9ede58cd083 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:21.821339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f98bbb-8471-4c60-bc07-d9ede58cd083 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:21.826283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 117/232] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:21 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:21.832943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f98bbb-8471-4c60-bc07-d9ede58cd083 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:21.910606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d436e-820a-4b24-9593-250a425dda35 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:21.938105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:21.988012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bc8f6-bbbf-4570-a6eb-b8299241623d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:21.992173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bc8f6-bbbf-4570-a6eb-b8299241623d None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:21.992786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bc8f6-bbbf-4570-a6eb-b8299241623d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:21.993264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bc8f6-bbbf-4570-a6eb-b8299241623d None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce/users/99b2b46dd8ef4441ad28e83a8205b76d/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:22.002279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bc8f6-bbbf-4570-a6eb-b8299241623d None admin] RBAC: Authorizing `identity:create_grant(project_id=908f423535c84512acd6b17bd7b971ce, user_id=99b2b46dd8ef4441ad28e83a8205b76d, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:22.004276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bc8f6-bbbf-4570-a6eb-b8299241623d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:22.028002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 116/233] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:07:21 2022] PUT /identity/v3/projects/908f423535c84512acd6b17bd7b971ce/users/99b2b46dd8ef4441ad28e83a8205b76d/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:07:22.050125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:22.058111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f98bbb-8471-4c60-bc07-d9ede58cd083 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:22.124139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf764df1-ac7b-4a0b-99d0-ac740a5842c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:22.130523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf764df1-ac7b-4a0b-99d0-ac740a5842c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:22.130831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf764df1-ac7b-4a0b-99d0-ac740a5842c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:22.131089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf764df1-ac7b-4a0b-99d0-ac740a5842c2 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce/users/99b2b46dd8ef4441ad28e83a8205b76d/roles/49053ad44c6c4cd1b932fcdb45321ff9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:22.133630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf764df1-ac7b-4a0b-99d0-ac740a5842c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=908f423535c84512acd6b17bd7b971ce, user_id=99b2b46dd8ef4441ad28e83a8205b76d, role_id=49053ad44c6c4cd1b932fcdb45321ff9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:22.135808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf764df1-ac7b-4a0b-99d0-ac740a5842c2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:22.157765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 118/234] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:07:21 2022] PUT /identity/v3/projects/908f423535c84512acd6b17bd7b971ce/users/99b2b46dd8ef4441ad28e83a8205b76d/roles/49053ad44c6c4cd1b932fcdb45321ff9 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:22.425342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b95e9c-5f1a-4051-8e44-58780d84a516 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:22.426030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b95e9c-5f1a-4051-8e44-58780d84a516 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:22.426521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b95e9c-5f1a-4051-8e44-58780d84a516 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:22.428663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 117/235] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:22.434336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8390f-5772-4dc0-a291-8617e1c04776 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:22.434836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8390f-5772-4dc0-a291-8617e1c04776 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:22.434910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8390f-5772-4dc0-a291-8617e1c04776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:22.462303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-49a8390f-5772-4dc0-a291-8617e1c04776 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:22.464090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-49a8390f-5772-4dc0-a291-8617e1c04776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:22.550072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 119/236] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:22.600811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b95e9c-5f1a-4051-8e44-58780d84a516 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:22.702076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413b3e6-b1b5-4fa3-998e-94941d6104df None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:22.702635 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413b3e6-b1b5-4fa3-998e-94941d6104df None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:22.703136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413b3e6-b1b5-4fa3-998e-94941d6104df None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:22.704990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 120/237] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:22.710799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccabd5b-516b-448e-8748-d6492fce5de9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:22.711376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccabd5b-516b-448e-8748-d6492fce5de9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:22.711860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccabd5b-516b-448e-8748-d6492fce5de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:22.731065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0ccabd5b-516b-448e-8748-d6492fce5de9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:22.732890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0ccabd5b-516b-448e-8748-d6492fce5de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:22.790430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b95e9c-5f1a-4051-8e44-58780d84a516 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:22.858275 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80724d-5706-4353-8647-db4e6a2c0d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:22.859089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80724d-5706-4353-8647-db4e6a2c0d18 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:22.859371 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80724d-5706-4353-8647-db4e6a2c0d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:22.859648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80724d-5706-4353-8647-db4e6a2c0d18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:22.862492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80724d-5706-4353-8647-db4e6a2c0d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:22.863975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80724d-5706-4353-8647-db4e6a2c0d18 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:22.866028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 118/238] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:07:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:22.871862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80724d-5706-4353-8647-db4e6a2c0d18 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:22.877196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 121/239] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:22.913775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccabd5b-516b-448e-8748-d6492fce5de9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:22.924865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:22.934994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:23.089257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80724d-5706-4353-8647-db4e6a2c0d18 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:23.125340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccabd5b-516b-448e-8748-d6492fce5de9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:23.181260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40c62b-359c-4011-af42-eeaf45c1786d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:23.182719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40c62b-359c-4011-af42-eeaf45c1786d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:23.183213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40c62b-359c-4011-af42-eeaf45c1786d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:23.183501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40c62b-359c-4011-af42-eeaf45c1786d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:23.199007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4e43f-a569-4e16-89a8-0e9febbdc443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:23.200146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4e43f-a569-4e16-89a8-0e9febbdc443 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:23.200644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4e43f-a569-4e16-89a8-0e9febbdc443 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:23.201136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4e43f-a569-4e16-89a8-0e9febbdc443 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:23.202538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4e43f-a569-4e16-89a8-0e9febbdc443 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:23.206213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4e43f-a569-4e16-89a8-0e9febbdc443 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:23.222204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40c62b-359c-4011-af42-eeaf45c1786d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-ae40c62b-359c-4011-af42-eeaf45c1786d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:23.231372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:23.232763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:23.233673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:23.239446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 122/240] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:23.245671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40c62b-359c-4011-af42-eeaf45c1786d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:23.259403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 119/241] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:07:22 2022] POST /identity/v3/users => generated 293 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:07:23.381734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:23.428641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb313ac2-5eaa-4c1e-8f15-99eeb0a1cc32 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:23.428982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb313ac2-5eaa-4c1e-8f15-99eeb0a1cc32 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:23.429216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb313ac2-5eaa-4c1e-8f15-99eeb0a1cc32 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:23.431082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 120/242] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:23.437821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36148-4557-4217-bafd-0bcbe8ee598e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:23.438123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36148-4557-4217-bafd-0bcbe8ee598e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:23.438389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36148-4557-4217-bafd-0bcbe8ee598e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:23.456502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40c62b-359c-4011-af42-eeaf45c1786d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:23.479119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e9c36148-4557-4217-bafd-0bcbe8ee598e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:23.480529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e9c36148-4557-4217-bafd-0bcbe8ee598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:23.528110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532894d-37b5-4f87-a518-c7a58cb368dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:23.528853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532894d-37b5-4f87-a518-c7a58cb368dd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:23.529148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532894d-37b5-4f87-a518-c7a58cb368dd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:23.530062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532894d-37b5-4f87-a518-c7a58cb368dd None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:23.530844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532894d-37b5-4f87-a518-c7a58cb368dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:23.531451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532894d-37b5-4f87-a518-c7a58cb368dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:23.533162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532894d-37b5-4f87-a518-c7a58cb368dd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:23.550213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 123/243] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:23.562225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532894d-37b5-4f87-a518-c7a58cb368dd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:23.629048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 121/244] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:23.654913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c36148-4557-4217-bafd-0bcbe8ee598e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:23.689315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:23.702886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532894d-37b5-4f87-a518-c7a58cb368dd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:23.742699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:23.761006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1b3c-d719-4297-bb92-6f01882cdbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:23.763218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1b3c-d719-4297-bb92-6f01882cdbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:23.763725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1b3c-d719-4297-bb92-6f01882cdbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:23.764129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1b3c-d719-4297-bb92-6f01882cdbe8 None admin] PATH_INFO: `/v3/users/86fba8a184714aaeb0f28ea7783aae9c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:23.800670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf1b3c-d719-4297-bb92-6f01882cdbe8 None admin] RBAC: Authorizing `identity:get_user(user_id=86fba8a184714aaeb0f28ea7783aae9c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:23.802771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf1b3c-d719-4297-bb92-6f01882cdbe8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:23.805147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 124/245] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:23 2022] GET /identity/v3/users/86fba8a184714aaeb0f28ea7783aae9c => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:23.812852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1b3c-d719-4297-bb92-6f01882cdbe8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:23.841272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c36148-4557-4217-bafd-0bcbe8ee598e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:23.894311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0163d-de11-4a22-b8d4-3edb448ac516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:23.895172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0163d-de11-4a22-b8d4-3edb448ac516 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:23.895452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0163d-de11-4a22-b8d4-3edb448ac516 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:23.895725 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0163d-de11-4a22-b8d4-3edb448ac516 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:23.909227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0163d-de11-4a22-b8d4-3edb448ac516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-abf0163d-de11-4a22-b8d4-3edb448ac516 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:23.918362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:23.919342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:23.920180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 122/246] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:07:23.925937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0163d-de11-4a22-b8d4-3edb448ac516 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:23.950245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1b3c-d719-4297-bb92-6f01882cdbe8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:23.991284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e378f4b-316f-4dab-bce6-25af05a9f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:23.992335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e378f4b-316f-4dab-bce6-25af05a9f13f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:23.992871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e378f4b-316f-4dab-bce6-25af05a9f13f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:23.993369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e378f4b-316f-4dab-bce6-25af05a9f13f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:24.051336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:24.067767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:24.084016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e378f4b-316f-4dab-bce6-25af05a9f13f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:24.084210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0163d-de11-4a22-b8d4-3edb448ac516 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:24.086048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e378f4b-316f-4dab-bce6-25af05a9f13f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:24.144730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d54a97-6727-4e7a-a176-7b5fb86fde7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:24.145952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d54a97-6727-4e7a-a176-7b5fb86fde7f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:24.146452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d54a97-6727-4e7a-a176-7b5fb86fde7f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:24.146944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d54a97-6727-4e7a-a176-7b5fb86fde7f None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:24.148703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d54a97-6727-4e7a-a176-7b5fb86fde7f None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:24.148834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d54a97-6727-4e7a-a176-7b5fb86fde7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:24.152098 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d54a97-6727-4e7a-a176-7b5fb86fde7f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:24.167791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 123/247] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:24.175800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d54a97-6727-4e7a-a176-7b5fb86fde7f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:24.215413 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 125/248] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:24.221045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e378f4b-316f-4dab-bce6-25af05a9f13f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:24.487844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:24.493680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d54a97-6727-4e7a-a176-7b5fb86fde7f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:24.540548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e378f4b-316f-4dab-bce6-25af05a9f13f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:24.587628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80708374-e2f9-4817-85f1-9ced9fdabfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:24.588922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80708374-e2f9-4817-85f1-9ced9fdabfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:24.589403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80708374-e2f9-4817-85f1-9ced9fdabfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:24.589869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80708374-e2f9-4817-85f1-9ced9fdabfd2 None admin] PATH_INFO: `/v3/users/bea64f92a6364520b93be53ce08ff4a9` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:24.591478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80708374-e2f9-4817-85f1-9ced9fdabfd2 None admin] RBAC: Authorizing `identity:get_user(user_id=bea64f92a6364520b93be53ce08ff4a9)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:24.593240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80708374-e2f9-4817-85f1-9ced9fdabfd2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:24.600715 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 124/249] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:24 2022] GET /identity/v3/users/bea64f92a6364520b93be53ce08ff4a9 => generated 297 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:24.609988 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80708374-e2f9-4817-85f1-9ced9fdabfd2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:24.624597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306a5b1-b639-4806-b249-be940db15619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:24.625506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306a5b1-b639-4806-b249-be940db15619 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:24.625824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306a5b1-b639-4806-b249-be940db15619 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:24.626104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306a5b1-b639-4806-b249-be940db15619 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:24.630711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306a5b1-b639-4806-b249-be940db15619 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:24.632529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306a5b1-b639-4806-b249-be940db15619 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:24.635647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 126/250] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:24 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:24.641516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306a5b1-b639-4806-b249-be940db15619 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:24.652936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:24.803586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80708374-e2f9-4817-85f1-9ced9fdabfd2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:24.825299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306a5b1-b639-4806-b249-be940db15619 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:24.845562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b422ca2-ea74-4c2a-a6c0-d756670fbfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:24.846722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422ca2-ea74-4c2a-a6c0-d756670fbfbc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:24.847127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422ca2-ea74-4c2a-a6c0-d756670fbfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:24.847522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422ca2-ea74-4c2a-a6c0-d756670fbfbc None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:24.849112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b422ca2-ea74-4c2a-a6c0-d756670fbfbc None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:24.850789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b422ca2-ea74-4c2a-a6c0-d756670fbfbc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:24.852823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 125/251] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:24 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:24.859432 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b422ca2-ea74-4c2a-a6c0-d756670fbfbc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:24.877852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab5eed-a77c-4604-bb6a-4f47f1aba936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:24.879277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab5eed-a77c-4604-bb6a-4f47f1aba936 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:24.879726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab5eed-a77c-4604-bb6a-4f47f1aba936 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:24.880410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab5eed-a77c-4604-bb6a-4f47f1aba936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:24.881439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab5eed-a77c-4604-bb6a-4f47f1aba936 None admin] RBAC: Adding query filter params (role.id=443ba7d155f54b0f85e995b760bd77bb, scope.project.id=908f423535c84512acd6b17bd7b971ce, user.id=86fba8a184714aaeb0f28ea7783aae9c) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:24.882152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab5eed-a77c-4604-bb6a-4f47f1aba936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:24.884266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab5eed-a77c-4604-bb6a-4f47f1aba936 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:24.893557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 127/252] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:24 2022] GET /identity/v3/role_assignments?user.id=86fba8a184714aaeb0f28ea7783aae9c&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=443ba7d155f54b0f85e995b760bd77bb => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:25.018456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:25.045377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b422ca2-ea74-4c2a-a6c0-d756670fbfbc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:25.097597 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3386a05-cce8-4745-97f9-45196112a46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:25.101451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386a05-cce8-4745-97f9-45196112a46b None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:25.101987 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386a05-cce8-4745-97f9-45196112a46b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:25.102413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386a05-cce8-4745-97f9-45196112a46b None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce/users/bea64f92a6364520b93be53ce08ff4a9/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:25.106309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3386a05-cce8-4745-97f9-45196112a46b None admin] RBAC: Authorizing `identity:create_grant(project_id=908f423535c84512acd6b17bd7b971ce, user_id=bea64f92a6364520b93be53ce08ff4a9, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:25.108326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3386a05-cce8-4745-97f9-45196112a46b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:25.135299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 126/253] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:07:24 2022] PUT /identity/v3/projects/908f423535c84512acd6b17bd7b971ce/users/bea64f92a6364520b93be53ce08ff4a9/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:07:25.715914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e09ae-6203-448a-aada-3bfdc4b14c08 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:25.716674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e09ae-6203-448a-aada-3bfdc4b14c08 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:25.716674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e09ae-6203-448a-aada-3bfdc4b14c08 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:25.718145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 128/254] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:25.730521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849f8d4-a263-4a68-a372-354776747735 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:25.731287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849f8d4-a263-4a68-a372-354776747735 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:25.731287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849f8d4-a263-4a68-a372-354776747735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:25.761396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a849f8d4-a263-4a68-a372-354776747735 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:25.763002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a849f8d4-a263-4a68-a372-354776747735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:25.861833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 127/255] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:25.940723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b00bd7-35d8-4c58-9d7b-6ba57fce7a13 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:25.941267 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b00bd7-35d8-4c58-9d7b-6ba57fce7a13 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:25.942080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b00bd7-35d8-4c58-9d7b-6ba57fce7a13 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:25.945236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 129/256] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:25.951798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf10fb-40df-458c-bf27-d746b3a1fd59 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:25.952513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf10fb-40df-458c-bf27-d746b3a1fd59 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:25.952705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b00bd7-35d8-4c58-9d7b-6ba57fce7a13 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:25.953255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf10fb-40df-458c-bf27-d746b3a1fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:26.027255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-cbbf10fb-40df-458c-bf27-d746b3a1fd59 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:26.031524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cbbf10fb-40df-458c-bf27-d746b3a1fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:26.039838 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:26.261119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 128/257] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:26.296095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf10fb-40df-458c-bf27-d746b3a1fd59 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:26.298551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:26.354895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b00bd7-35d8-4c58-9d7b-6ba57fce7a13 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:26.490956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6460fa2-8928-4d73-bd11-da6ecba1629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:26.492237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6460fa2-8928-4d73-bd11-da6ecba1629a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:26.492692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6460fa2-8928-4d73-bd11-da6ecba1629a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:26.493163 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6460fa2-8928-4d73-bd11-da6ecba1629a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:26.533864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6460fa2-8928-4d73-bd11-da6ecba1629a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f6460fa2-8928-4d73-bd11-da6ecba1629a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:26.553853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:26.555497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:26.557145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:26.567233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 130/258] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:07:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 615 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:26.576242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6460fa2-8928-4d73-bd11-da6ecba1629a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:26.610170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf10fb-40df-458c-bf27-d746b3a1fd59 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:26.723728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e33361-c75e-471e-89a3-27428d7d6128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:26.725441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e33361-c75e-471e-89a3-27428d7d6128 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:26.726056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e33361-c75e-471e-89a3-27428d7d6128 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:26.726648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e33361-c75e-471e-89a3-27428d7d6128 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:26.747144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e33361-c75e-471e-89a3-27428d7d6128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-80e33361-c75e-471e-89a3-27428d7d6128 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:26.765547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:26.767309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:26.768820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:26.773496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 129/259] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 478 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:07:26.779970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e33361-c75e-471e-89a3-27428d7d6128 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:26.848074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:26.908481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6460fa2-8928-4d73-bd11-da6ecba1629a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:26.949475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:26.973686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268ea3f-d9d5-4122-bc18-6b2a6b9fb967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:26.974388 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268ea3f-d9d5-4122-bc18-6b2a6b9fb967 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:26.974665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268ea3f-d9d5-4122-bc18-6b2a6b9fb967 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:26.975105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268ea3f-d9d5-4122-bc18-6b2a6b9fb967 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:26.975999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268ea3f-d9d5-4122-bc18-6b2a6b9fb967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:26.976443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268ea3f-d9d5-4122-bc18-6b2a6b9fb967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:26.978006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268ea3f-d9d5-4122-bc18-6b2a6b9fb967 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:26.988225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 131/260] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:27.009484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268ea3f-d9d5-4122-bc18-6b2a6b9fb967 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:27.041471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e33361-c75e-471e-89a3-27428d7d6128 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:27.137674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb2fac-9476-4af0-b406-f143b43b8fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:27.138612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb2fac-9476-4af0-b406-f143b43b8fdf None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:27.139058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb2fac-9476-4af0-b406-f143b43b8fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:27.139473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb2fac-9476-4af0-b406-f143b43b8fdf None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:27.140555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb2fac-9476-4af0-b406-f143b43b8fdf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:27.141105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb2fac-9476-4af0-b406-f143b43b8fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:27.142659 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb2fac-9476-4af0-b406-f143b43b8fdf None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:27.159757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 130/261] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:27.165437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb2fac-9476-4af0-b406-f143b43b8fdf None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:27.261529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268ea3f-d9d5-4122-bc18-6b2a6b9fb967 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:27.383909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261f773-fc4a-41ba-b291-5513d330e019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:27.385414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261f773-fc4a-41ba-b291-5513d330e019 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:27.385906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261f773-fc4a-41ba-b291-5513d330e019 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:27.386194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261f773-fc4a-41ba-b291-5513d330e019 None admin] PATH_INFO: `/v3/users/99b2b46dd8ef4441ad28e83a8205b76d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:27.390769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261f773-fc4a-41ba-b291-5513d330e019 None admin] RBAC: Authorizing `identity:get_user(user_id=99b2b46dd8ef4441ad28e83a8205b76d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:27.392252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261f773-fc4a-41ba-b291-5513d330e019 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:27.394102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 132/262] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:27 2022] GET /identity/v3/users/99b2b46dd8ef4441ad28e83a8205b76d => generated 283 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:27.396644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b060a-beb1-4272-a8cd-7c269ca9e36c None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:27.397160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b060a-beb1-4272-a8cd-7c269ca9e36c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:27.397589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b060a-beb1-4272-a8cd-7c269ca9e36c None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:27.399545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 133/263] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:27.406631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08664af-741a-4c77-8bfe-efa00cb1d9b7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:27.407214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08664af-741a-4c77-8bfe-efa00cb1d9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:27.407648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08664af-741a-4c77-8bfe-efa00cb1d9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:27.430608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c08664af-741a-4c77-8bfe-efa00cb1d9b7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:27.432106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c08664af-741a-4c77-8bfe-efa00cb1d9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:27.434729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:27.442557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb2fac-9476-4af0-b406-f143b43b8fdf None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:27.469785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:27.498157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64274979-f7f1-4e9b-831c-0ed4742ad161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:27.501429 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64274979-f7f1-4e9b-831c-0ed4742ad161 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:27.501969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64274979-f7f1-4e9b-831c-0ed4742ad161 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:27.502464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64274979-f7f1-4e9b-831c-0ed4742ad161 None admin] PATH_INFO: `/v3/users/99b2b46dd8ef4441ad28e83a8205b76d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:27.506570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64274979-f7f1-4e9b-831c-0ed4742ad161 None admin] RBAC: Authorizing `identity:get_user(user_id=99b2b46dd8ef4441ad28e83a8205b76d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:27.508340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64274979-f7f1-4e9b-831c-0ed4742ad161 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:27.511595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 131/264] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:27 2022] GET /identity/v3/users/99b2b46dd8ef4441ad28e83a8205b76d => generated 283 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:27.512842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64274979-f7f1-4e9b-831c-0ed4742ad161 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:27.570834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 134/265] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:27.574201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08664af-741a-4c77-8bfe-efa00cb1d9b7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:27.730351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64274979-f7f1-4e9b-831c-0ed4742ad161 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:27.769023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08664af-741a-4c77-8bfe-efa00cb1d9b7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:27.791804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918ae71-eb06-4cd5-9491-4377a18fdb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:27.793083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918ae71-eb06-4cd5-9491-4377a18fdb1b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:27.793507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918ae71-eb06-4cd5-9491-4377a18fdb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:27.793945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918ae71-eb06-4cd5-9491-4377a18fdb1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:27.816058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b9fb6-3476-4a0b-a7ea-819a5b8a3837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:27.816993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9fb6-3476-4a0b-a7ea-819a5b8a3837 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:27.817424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9fb6-3476-4a0b-a7ea-819a5b8a3837 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:27.817885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9fb6-3476-4a0b-a7ea-819a5b8a3837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:27.828003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:27.882264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:27.896887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918ae71-eb06-4cd5-9491-4377a18fdb1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:27.898682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918ae71-eb06-4cd5-9491-4377a18fdb1b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:27.902865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b9fb6-3476-4a0b-a7ea-819a5b8a3837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:27.904936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b9fb6-3476-4a0b-a7ea-819a5b8a3837 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:28.013375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 135/266] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:28.014747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b9fb6-3476-4a0b-a7ea-819a5b8a3837 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:28.020073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 132/267] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:28.021522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc382d29-7139-4dd8-a1d9-058bc2caa74c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:28.022019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc382d29-7139-4dd8-a1d9-058bc2caa74c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:28.022446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc382d29-7139-4dd8-a1d9-058bc2caa74c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:28.023922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 133/268] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:28.024849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc382d29-7139-4dd8-a1d9-058bc2caa74c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:28.064179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:28.138378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b9fb6-3476-4a0b-a7ea-819a5b8a3837 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:28.142793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:28.155363 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc382d29-7139-4dd8-a1d9-058bc2caa74c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:28.182289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54961c-fabf-422a-a143-3275da66b9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:28.183694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54961c-fabf-422a-a143-3275da66b9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:28.184228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54961c-fabf-422a-a143-3275da66b9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:28.184708 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54961c-fabf-422a-a143-3275da66b9a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:28.198453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54961c-fabf-422a-a143-3275da66b9a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:28.200497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb497b-d95d-4713-94d5-a937bda7bea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:28.201394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb497b-d95d-4713-94d5-a937bda7bea4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:28.201679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb497b-d95d-4713-94d5-a937bda7bea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:28.201991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb497b-d95d-4713-94d5-a937bda7bea4 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:28.203361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb497b-d95d-4713-94d5-a937bda7bea4 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:28.205320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb497b-d95d-4713-94d5-a937bda7bea4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:28.207233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 134/269] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:28 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:28.208049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb497b-d95d-4713-94d5-a937bda7bea4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-4b54961c-fabf-422a-a143-3275da66b9a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:28.209354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:28.210278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:28.211083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:28.214312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 136/270] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:28.215924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0303f5-67b0-41d8-b264-7599504b28e1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:28.216455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0303f5-67b0-41d8-b264-7599504b28e1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:28.216927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0303f5-67b0-41d8-b264-7599504b28e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:28.246706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ee0303f5-67b0-41d8-b264-7599504b28e1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:28.248304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ee0303f5-67b0-41d8-b264-7599504b28e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:28.349075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 137/271] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:28.358283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0303f5-67b0-41d8-b264-7599504b28e1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:28.381658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb497b-d95d-4713-94d5-a937bda7bea4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:28.398668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:28.424320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab266b5-a652-42d0-861e-5b77f313171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:28.425201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab266b5-a652-42d0-861e-5b77f313171a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:28.425516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab266b5-a652-42d0-861e-5b77f313171a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:28.425939 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab266b5-a652-42d0-861e-5b77f313171a None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:28.427286 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab266b5-a652-42d0-861e-5b77f313171a None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:28.428940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab266b5-a652-42d0-861e-5b77f313171a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:28.431183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 135/272] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:28 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:28.433021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab266b5-a652-42d0-861e-5b77f313171a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:28.474161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:28.527351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0303f5-67b0-41d8-b264-7599504b28e1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:28.583241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab266b5-a652-42d0-861e-5b77f313171a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:28.601196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85246e4-a6cc-4366-bf82-8ce7a028666b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:28.602793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85246e4-a6cc-4366-bf82-8ce7a028666b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:28.604079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85246e4-a6cc-4366-bf82-8ce7a028666b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:28.605498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85246e4-a6cc-4366-bf82-8ce7a028666b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:28.607416 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85246e4-a6cc-4366-bf82-8ce7a028666b None admin] RBAC: Adding query filter params (role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=908f423535c84512acd6b17bd7b971ce, user.id=99b2b46dd8ef4441ad28e83a8205b76d) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:28.608456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85246e4-a6cc-4366-bf82-8ce7a028666b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:28.611485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85246e4-a6cc-4366-bf82-8ce7a028666b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:28.625860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 138/273] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:28 2022] GET /identity/v3/role_assignments?user.id=99b2b46dd8ef4441ad28e83a8205b76d&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:28.627651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85246e4-a6cc-4366-bf82-8ce7a028666b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:28.687562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2184-1498-400b-aca9-6bf8820b745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:28.689186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2184-1498-400b-aca9-6bf8820b745b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:28.691886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2184-1498-400b-aca9-6bf8820b745b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:28.692622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2184-1498-400b-aca9-6bf8820b745b None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:28.694291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af2184-1498-400b-aca9-6bf8820b745b None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:28.698643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af2184-1498-400b-aca9-6bf8820b745b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:28.705786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af2184-1498-400b-aca9-6bf8820b745b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:28.732303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 136/274] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:28.734500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2184-1498-400b-aca9-6bf8820b745b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:28.862058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:28.889608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85246e4-a6cc-4366-bf82-8ce7a028666b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:28.927625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:28.944182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2184-1498-400b-aca9-6bf8820b745b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:28.989387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be19c0b-635b-482a-8ba0-f0452a257aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:28.991956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be19c0b-635b-482a-8ba0-f0452a257aea None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:28.992876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be19c0b-635b-482a-8ba0-f0452a257aea None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:28.993536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be19c0b-635b-482a-8ba0-f0452a257aea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:28.998018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be19c0b-635b-482a-8ba0-f0452a257aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:29.001195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be19c0b-635b-482a-8ba0-f0452a257aea None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:29.005092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1d41f-e5f9-475a-a4b0-dce42a025a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:29.005410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 139/275] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:07:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:29.007858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1d41f-e5f9-475a-a4b0-dce42a025a94 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.007858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1d41f-e5f9-475a-a4b0-dce42a025a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.007858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1d41f-e5f9-475a-a4b0-dce42a025a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.007858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1d41f-e5f9-475a-a4b0-dce42a025a94 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=908f423535c84512acd6b17bd7b971ce, user.id=99b2b46dd8ef4441ad28e83a8205b76d) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:29.012728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1d41f-e5f9-475a-a4b0-dce42a025a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:29.012728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1d41f-e5f9-475a-a4b0-dce42a025a94 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:29.013543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45624da-00bd-4308-894f-2135cc064557 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.014286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45624da-00bd-4308-894f-2135cc064557 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.014954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45624da-00bd-4308-894f-2135cc064557 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.017273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 140/276] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:29.018712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45624da-00bd-4308-894f-2135cc064557 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:29.028941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 137/277] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:28 2022] GET /identity/v3/role_assignments?user.id=99b2b46dd8ef4441ad28e83a8205b76d&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:29.030076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1d41f-e5f9-475a-a4b0-dce42a025a94 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:29.216819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45624da-00bd-4308-894f-2135cc064557 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:29.220885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1d41f-e5f9-475a-a4b0-dce42a025a94 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:29.279133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57b1ee-da9a-4504-9991-8c5b79c30295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:29.280190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b1ee-da9a-4504-9991-8c5b79c30295 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.280460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b1ee-da9a-4504-9991-8c5b79c30295 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.280730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b1ee-da9a-4504-9991-8c5b79c30295 None admin] PATH_INFO: `/v3/users/ee1bd4f393c540f9885a88a4f10d2f47` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.287420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497140e-0bf5-4c4f-b322-9419cdd4c575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:29.288570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497140e-0bf5-4c4f-b322-9419cdd4c575 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.289049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497140e-0bf5-4c4f-b322-9419cdd4c575 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.289539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497140e-0bf5-4c4f-b322-9419cdd4c575 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.291110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497140e-0bf5-4c4f-b322-9419cdd4c575 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:29.294174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497140e-0bf5-4c4f-b322-9419cdd4c575 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:29.300698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57b1ee-da9a-4504-9991-8c5b79c30295 None admin] RBAC: Authorizing `identity:get_user(user_id=ee1bd4f393c540f9885a88a4f10d2f47)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:29.302316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57b1ee-da9a-4504-9991-8c5b79c30295 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:29.312147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 141/278] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:29 2022] GET /identity/v3/users/ee1bd4f393c540f9885a88a4f10d2f47 => generated 293 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:29.312147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0d9e9-dece-4608-b75d-4fe8e4a4a40a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.312147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0d9e9-dece-4608-b75d-4fe8e4a4a40a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.312147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0d9e9-dece-4608-b75d-4fe8e4a4a40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.331139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c8e0d9e9-dece-4608-b75d-4fe8e4a4a40a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:29.333579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c8e0d9e9-dece-4608-b75d-4fe8e4a4a40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:29.340382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 138/279] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:07:29 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:07:29.341615 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a587130-1850-4f79-8b5e-64a705aa2a23 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.341937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a587130-1850-4f79-8b5e-64a705aa2a23 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.342641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a587130-1850-4f79-8b5e-64a705aa2a23 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.343960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 139/280] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:29.344545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a587130-1850-4f79-8b5e-64a705aa2a23 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:29.365766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:29.370511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:29.476492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 142/281] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:29.478887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f49db-4e51-4254-b465-528d0ffb3091 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.479179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f49db-4e51-4254-b465-528d0ffb3091 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.479545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f49db-4e51-4254-b465-528d0ffb3091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.503523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1d3f49db-4e51-4254-b465-528d0ffb3091 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:29.504800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1d3f49db-4e51-4254-b465-528d0ffb3091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:29.539343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a587130-1850-4f79-8b5e-64a705aa2a23 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:29.593129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c83f91-d5fb-466f-bd9d-e0862bb3ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:29.594219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c83f91-d5fb-466f-bd9d-e0862bb3ea56 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.594503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c83f91-d5fb-466f-bd9d-e0862bb3ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.594775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c83f91-d5fb-466f-bd9d-e0862bb3ea56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.595738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c83f91-d5fb-466f-bd9d-e0862bb3ea56 None admin] RBAC: Adding query filter params (role.id=4a06d59562de4d598671a462e3eaf62a, user.id=ee1bd4f393c540f9885a88a4f10d2f47, scope.system=all) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:29.596276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c83f91-d5fb-466f-bd9d-e0862bb3ea56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:29.598317 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c83f91-d5fb-466f-bd9d-e0862bb3ea56 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:29.606326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 140/282] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Jun 6 14:07:29 2022] GET /identity/v3/role_assignments?user.id=ee1bd4f393c540f9885a88a4f10d2f47&scope.system=all&role.id=4a06d59562de4d598671a462e3eaf62a => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:29.606699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c83f91-d5fb-466f-bd9d-e0862bb3ea56 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:29.620633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 143/283] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:29.649839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f49db-4e51-4254-b465-528d0ffb3091 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:29.760837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:29.813612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c83f91-d5fb-466f-bd9d-e0862bb3ea56 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:29.838815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:29.868814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f5017-460d-40e4-9cde-d5c4ae37e406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:29.870159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5017-460d-40e4-9cde-d5c4ae37e406 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.870675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5017-460d-40e4-9cde-d5c4ae37e406 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.871196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5017-460d-40e4-9cde-d5c4ae37e406 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.876926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f49db-4e51-4254-b465-528d0ffb3091 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:29.891600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f5017-460d-40e4-9cde-d5c4ae37e406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f90f5017-460d-40e4-9cde-d5c4ae37e406 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:29.906552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:29.907580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:29.908480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:29.909173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 141/284] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:29.919426 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f5017-460d-40e4-9cde-d5c4ae37e406 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:29.950445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12b8d0-6d59-427a-b5e3-21501b8c2262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:29.951268 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12b8d0-6d59-427a-b5e3-21501b8c2262 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:29.951550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12b8d0-6d59-427a-b5e3-21501b8c2262 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:29.951825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12b8d0-6d59-427a-b5e3-21501b8c2262 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:29.972016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12b8d0-6d59-427a-b5e3-21501b8c2262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-5c12b8d0-6d59-427a-b5e3-21501b8c2262 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:29.980071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:29.981155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:29.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 144/285] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:29.991611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12b8d0-6d59-427a-b5e3-21501b8c2262 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:30.193952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f5017-460d-40e4-9cde-d5c4ae37e406 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:30.195168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12b8d0-6d59-427a-b5e3-21501b8c2262 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:30.255128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4d27a-fb6e-48e6-b035-01b96b4ddf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:30.258140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4d27a-fb6e-48e6-b035-01b96b4ddf0d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:30.258622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4d27a-fb6e-48e6-b035-01b96b4ddf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:30.259106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4d27a-fb6e-48e6-b035-01b96b4ddf0d None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:30.260638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4d27a-fb6e-48e6-b035-01b96b4ddf0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:30.261180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4d27a-fb6e-48e6-b035-01b96b4ddf0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:30.270564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4d27a-fb6e-48e6-b035-01b96b4ddf0d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:30.278675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:30.305746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 142/286] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:30.314764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4d27a-fb6e-48e6-b035-01b96b4ddf0d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:30.315183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60db3-7449-4355-b721-3c35f47327cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:30.315807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60db3-7449-4355-b721-3c35f47327cd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:30.316200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60db3-7449-4355-b721-3c35f47327cd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:30.316485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60db3-7449-4355-b721-3c35f47327cd None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:30.317271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba60db3-7449-4355-b721-3c35f47327cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:30.317578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba60db3-7449-4355-b721-3c35f47327cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:30.319149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba60db3-7449-4355-b721-3c35f47327cd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:30.327023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:30.331484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 145/287] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:30.341320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60db3-7449-4355-b721-3c35f47327cd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:30.511800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60db3-7449-4355-b721-3c35f47327cd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:30.563225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ff881-ae80-4211-bf15-bf6f0306c7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:30.564509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ff881-ae80-4211-bf15-bf6f0306c7fd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:30.564997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ff881-ae80-4211-bf15-bf6f0306c7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:30.565511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ff881-ae80-4211-bf15-bf6f0306c7fd None admin] PATH_INFO: `/v3/users/bea64f92a6364520b93be53ce08ff4a9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:30.567071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ff881-ae80-4211-bf15-bf6f0306c7fd None admin] RBAC: Authorizing `identity:get_user(user_id=bea64f92a6364520b93be53ce08ff4a9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:30.569269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ff881-ae80-4211-bf15-bf6f0306c7fd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:30.571798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 146/288] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:30 2022] GET /identity/v3/users/bea64f92a6364520b93be53ce08ff4a9 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:30.583050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ff881-ae80-4211-bf15-bf6f0306c7fd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:30.667151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:30.670924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4d27a-fb6e-48e6-b035-01b96b4ddf0d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:30.711492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ff881-ae80-4211-bf15-bf6f0306c7fd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:30.754607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b43b4-26c4-420b-9475-68888c2ae53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:30.755650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b43b4-26c4-420b-9475-68888c2ae53d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:30.756139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b43b4-26c4-420b-9475-68888c2ae53d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:30.756642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b43b4-26c4-420b-9475-68888c2ae53d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:30.823007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc76fe-d76f-4c80-bc1b-c7c55f7e9529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:30.824653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc76fe-d76f-4c80-bc1b-c7c55f7e9529 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:30.825021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc76fe-d76f-4c80-bc1b-c7c55f7e9529 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:30.825506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc76fe-d76f-4c80-bc1b-c7c55f7e9529 None admin] PATH_INFO: `/v3/users/86fba8a184714aaeb0f28ea7783aae9c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:30.830999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc76fe-d76f-4c80-bc1b-c7c55f7e9529 None admin] RBAC: Authorizing `identity:get_user(user_id=86fba8a184714aaeb0f28ea7783aae9c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:30.833568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc76fe-d76f-4c80-bc1b-c7c55f7e9529 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:30.837140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 143/289] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:30 2022] GET /identity/v3/users/86fba8a184714aaeb0f28ea7783aae9c => generated 297 bytes in 524 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:30.841894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b43b4-26c4-420b-9475-68888c2ae53d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:30.843792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b43b4-26c4-420b-9475-68888c2ae53d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:30.846041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc76fe-d76f-4c80-bc1b-c7c55f7e9529 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:30.926715 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:30.982958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 147/290] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:30.991723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b43b4-26c4-420b-9475-68888c2ae53d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:31.066248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:31.115335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b43b4-26c4-420b-9475-68888c2ae53d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:31.156993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839946a-080b-4567-a4c6-688ec944325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:31.158060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839946a-080b-4567-a4c6-688ec944325c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:31.158447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839946a-080b-4567-a4c6-688ec944325c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:31.159338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839946a-080b-4567-a4c6-688ec944325c None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:31.161001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839946a-080b-4567-a4c6-688ec944325c None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:31.162818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839946a-080b-4567-a4c6-688ec944325c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:31.165189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 148/291] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:30 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:31.174672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839946a-080b-4567-a4c6-688ec944325c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:31.177242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc76fe-d76f-4c80-bc1b-c7c55f7e9529 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:31.227882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:31.228142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5524c1-d756-499a-a926-20a51d6a10bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:31.230383 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5524c1-d756-499a-a926-20a51d6a10bd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:31.230871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5524c1-d756-499a-a926-20a51d6a10bd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:31.231357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5524c1-d756-499a-a926-20a51d6a10bd None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:31.234161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5524c1-d756-499a-a926-20a51d6a10bd None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:31.236951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5524c1-d756-499a-a926-20a51d6a10bd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:31.240731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 144/292] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:30 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:31.249525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5524c1-d756-499a-a926-20a51d6a10bd None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:31.340240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839946a-080b-4567-a4c6-688ec944325c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:31.443136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa45ba-be0b-4869-bfd4-10502ef52299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:31.443710 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa45ba-be0b-4869-bfd4-10502ef52299 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:31.443995 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa45ba-be0b-4869-bfd4-10502ef52299 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:31.444273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa45ba-be0b-4869-bfd4-10502ef52299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:31.445222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa45ba-be0b-4869-bfd4-10502ef52299 None admin] RBAC: Adding query filter params (role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=908f423535c84512acd6b17bd7b971ce, user.id=bea64f92a6364520b93be53ce08ff4a9) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:31.445533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa45ba-be0b-4869-bfd4-10502ef52299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:31.447146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa45ba-be0b-4869-bfd4-10502ef52299 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:31.449549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5524c1-d756-499a-a926-20a51d6a10bd None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:31.473422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 149/293] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:31 2022] GET /identity/v3/role_assignments?user.id=bea64f92a6364520b93be53ce08ff4a9&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=4a06d59562de4d598671a462e3eaf62a => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:31.554676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:31.572269 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcc5ff-b556-4611-ab26-c339fe31c3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:31.573882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcc5ff-b556-4611-ab26-c339fe31c3dc None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:31.574382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcc5ff-b556-4611-ab26-c339fe31c3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:31.574874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcc5ff-b556-4611-ab26-c339fe31c3dc None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce/users/86fba8a184714aaeb0f28ea7783aae9c/roles/443ba7d155f54b0f85e995b760bd77bb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:31.584172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcc5ff-b556-4611-ab26-c339fe31c3dc None admin] RBAC: Authorizing `identity:create_grant(project_id=908f423535c84512acd6b17bd7b971ce, user_id=86fba8a184714aaeb0f28ea7783aae9c, role_id=443ba7d155f54b0f85e995b760bd77bb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:31.586786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcc5ff-b556-4611-ab26-c339fe31c3dc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:31.613240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 145/294] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:07:31 2022] PUT /identity/v3/projects/908f423535c84512acd6b17bd7b971ce/users/86fba8a184714aaeb0f28ea7783aae9c/roles/443ba7d155f54b0f85e995b760bd77bb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:32.422369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff28356-ceda-4a8e-b5a6-7e5c84f72af5 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:32.422917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff28356-ceda-4a8e-b5a6-7e5c84f72af5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:32.423196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff28356-ceda-4a8e-b5a6-7e5c84f72af5 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:32.424499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 150/295] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:32.430636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ddf4a-a8aa-4c32-86fb-1eae52e48771 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:32.431154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ddf4a-a8aa-4c32-86fb-1eae52e48771 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:32.431879 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ddf4a-a8aa-4c32-86fb-1eae52e48771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:32.452646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4e0ddf4a-a8aa-4c32-86fb-1eae52e48771 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:32.453958 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4e0ddf4a-a8aa-4c32-86fb-1eae52e48771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:32.566322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 146/296] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:32.591525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff28356-ceda-4a8e-b5a6-7e5c84f72af5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:32.741869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:32.747582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff28356-ceda-4a8e-b5a6-7e5c84f72af5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:32.792536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a2206-7219-48af-9399-b75f4ab475da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:32.793352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a2206-7219-48af-9399-b75f4ab475da None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:32.793629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a2206-7219-48af-9399-b75f4ab475da None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:32.793938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a2206-7219-48af-9399-b75f4ab475da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:32.807051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a2206-7219-48af-9399-b75f4ab475da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7e4a2206-7219-48af-9399-b75f4ab475da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:32.816067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:32.817089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:32.817955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 151/297] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:32.824814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ddf4a-a8aa-4c32-86fb-1eae52e48771 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:32.949506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ddf4a-a8aa-4c32-86fb-1eae52e48771 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:32.978120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:32.993111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a63d5c-da6f-4dce-90c3-c53087349e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:32.994731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a63d5c-da6f-4dce-90c3-c53087349e5f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:32.995809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a63d5c-da6f-4dce-90c3-c53087349e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:32.996357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a63d5c-da6f-4dce-90c3-c53087349e5f None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:32.997423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a63d5c-da6f-4dce-90c3-c53087349e5f None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:32.997996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a63d5c-da6f-4dce-90c3-c53087349e5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:32.999814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a63d5c-da6f-4dce-90c3-c53087349e5f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:33.011453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 147/298] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:33.016256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c26723-e715-4907-b96e-b00a8483e113 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.016620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c26723-e715-4907-b96e-b00a8483e113 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.016802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c26723-e715-4907-b96e-b00a8483e113 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.018565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 152/299] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:33.021258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a63d5c-da6f-4dce-90c3-c53087349e5f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:33.024203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f3b5-41f0-4d9a-9dc1-d1259de63247 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.024690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f3b5-41f0-4d9a-9dc1-d1259de63247 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.024981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f3b5-41f0-4d9a-9dc1-d1259de63247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.044188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1e22f3b5-41f0-4d9a-9dc1-d1259de63247 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:33.045424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1e22f3b5-41f0-4d9a-9dc1-d1259de63247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:33.125777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 153/300] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:33.131310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f7d50-ad19-4d9a-bf0e-2039bef096e4 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.131637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f7d50-ad19-4d9a-bf0e-2039bef096e4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.131876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f7d50-ad19-4d9a-bf0e-2039bef096e4 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.133473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 154/301] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:33.140175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ab3c5-e1d3-47f0-9ecc-17331095fa0a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.140449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ab3c5-e1d3-47f0-9ecc-17331095fa0a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.140724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ab3c5-e1d3-47f0-9ecc-17331095fa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.152562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a63d5c-da6f-4dce-90c3-c53087349e5f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:33.158885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ff8ab3c5-e1d3-47f0-9ecc-17331095fa0a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:33.160124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ff8ab3c5-e1d3-47f0-9ecc-17331095fa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:33.193970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea800de2-9179-4632-a2e6-8239e251413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:33.194859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea800de2-9179-4632-a2e6-8239e251413d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.195135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea800de2-9179-4632-a2e6-8239e251413d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.195403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea800de2-9179-4632-a2e6-8239e251413d None admin] PATH_INFO: `/v3/users/ee1bd4f393c540f9885a88a4f10d2f47` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.196720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea800de2-9179-4632-a2e6-8239e251413d None admin] RBAC: Authorizing `identity:get_user(user_id=ee1bd4f393c540f9885a88a4f10d2f47)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:33.198266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea800de2-9179-4632-a2e6-8239e251413d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:33.205103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 148/302] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:33 2022] GET /identity/v3/users/ee1bd4f393c540f9885a88a4f10d2f47 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:33.205964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea800de2-9179-4632-a2e6-8239e251413d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:33.249822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 155/303] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:33.249822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ab3c5-e1d3-47f0-9ecc-17331095fa0a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:33.267738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:33.287668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:33.346708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea800de2-9179-4632-a2e6-8239e251413d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:33.364816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ab3c5-e1d3-47f0-9ecc-17331095fa0a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:33.388479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4c6b3-2dfe-4ed3-a7bd-2ba71faad34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:33.389289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4c6b3-2dfe-4ed3-a7bd-2ba71faad34e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.389572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4c6b3-2dfe-4ed3-a7bd-2ba71faad34e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.389883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4c6b3-2dfe-4ed3-a7bd-2ba71faad34e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.391160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4c6b3-2dfe-4ed3-a7bd-2ba71faad34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:33.392677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4c6b3-2dfe-4ed3-a7bd-2ba71faad34e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:33.398999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 149/304] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:07:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:33.398999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4c6b3-2dfe-4ed3-a7bd-2ba71faad34e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:33.415370 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54156de-a136-4939-b48a-a476926a9c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:33.416898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54156de-a136-4939-b48a-a476926a9c15 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.417184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54156de-a136-4939-b48a-a476926a9c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.417463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54156de-a136-4939-b48a-a476926a9c15 None admin] PATH_INFO: `/v3/system/users/ee1bd4f393c540f9885a88a4f10d2f47/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.419185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54156de-a136-4939-b48a-a476926a9c15 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ee1bd4f393c540f9885a88a4f10d2f47, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:33.420618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54156de-a136-4939-b48a-a476926a9c15 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:33.435097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 156/305] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Jun 6 14:07:33 2022] PUT /identity/v3/system/users/ee1bd4f393c540f9885a88a4f10d2f47/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:33.436062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54156de-a136-4939-b48a-a476926a9c15 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:33.541680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4c6b3-2dfe-4ed3-a7bd-2ba71faad34e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:33.551959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:33.575631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:33.578034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54156de-a136-4939-b48a-a476926a9c15 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:33.595091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2ae76-a6ee-41d8-ab78-b4f5aaa2aab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:33.596407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2ae76-a6ee-41d8-ab78-b4f5aaa2aab8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.597398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2ae76-a6ee-41d8-ab78-b4f5aaa2aab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.597886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2ae76-a6ee-41d8-ab78-b4f5aaa2aab8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.617915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2ae76-a6ee-41d8-ab78-b4f5aaa2aab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:33.626060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f588be7-77a5-4c14-bf18-5fb9c01d786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:33.626907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f588be7-77a5-4c14-bf18-5fb9c01d786b None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.627265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f588be7-77a5-4c14-bf18-5fb9c01d786b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.627607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f588be7-77a5-4c14-bf18-5fb9c01d786b None admin] PATH_INFO: `/v3/groups` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.628436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f588be7-77a5-4c14-bf18-5fb9c01d786b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-3ba2ae76-a6ee-41d8-ab78-b4f5aaa2aab8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:33.629770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:33.630898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:33.631914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:33.632772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f588be7-77a5-4c14-bf18-5fb9c01d786b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:33.639810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 150/306] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:33.640777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2ae76-a6ee-41d8-ab78-b4f5aaa2aab8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:33.657243 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 157/307] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Jun 6 14:07:33 2022] POST /identity/v3/groups => generated 229 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:07:33.783541 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2ae76-a6ee-41d8-ab78-b4f5aaa2aab8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:33.829994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb3fcc-217a-4b4c-a34b-3643bcd40492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:33.830707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb3fcc-217a-4b4c-a34b-3643bcd40492 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:33.830995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb3fcc-217a-4b4c-a34b-3643bcd40492 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:33.831270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb3fcc-217a-4b4c-a34b-3643bcd40492 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:33.832071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb3fcc-217a-4b4c-a34b-3643bcd40492 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:33.832379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb3fcc-217a-4b4c-a34b-3643bcd40492 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:33.834076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb3fcc-217a-4b4c-a34b-3643bcd40492 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:33.845574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 151/308] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:33.851246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f588be7-77a5-4c14-bf18-5fb9c01d786b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:33.974727 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f588be7-77a5-4c14-bf18-5fb9c01d786b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:34.015133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2d621-04bc-4b90-8070-b49735e8b5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:34.016104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d621-04bc-4b90-8070-b49735e8b5ac None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:34.016386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d621-04bc-4b90-8070-b49735e8b5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:34.016654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d621-04bc-4b90-8070-b49735e8b5ac None admin] PATH_INFO: `/v3/users/be276d2d38d244dd989ccfdad02efb27` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:34.033153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2d621-04bc-4b90-8070-b49735e8b5ac None admin] RBAC: Authorizing `identity:get_user(user_id=be276d2d38d244dd989ccfdad02efb27)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:34.034692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2d621-04bc-4b90-8070-b49735e8b5ac None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:34.036475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 158/309] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:33 2022] GET /identity/v3/users/be276d2d38d244dd989ccfdad02efb27 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:34.042889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb3fcc-217a-4b4c-a34b-3643bcd40492 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:34.096061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:34.169117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb3fcc-217a-4b4c-a34b-3643bcd40492 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:34.213263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7487318-6688-4535-b1bb-5a5b4749e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:34.214035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7487318-6688-4535-b1bb-5a5b4749e911 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:34.214379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7487318-6688-4535-b1bb-5a5b4749e911 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:34.214729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7487318-6688-4535-b1bb-5a5b4749e911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:34.215679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7487318-6688-4535-b1bb-5a5b4749e911 None admin] RBAC: Adding query filter params (role.id=443ba7d155f54b0f85e995b760bd77bb, user.id=be276d2d38d244dd989ccfdad02efb27, scope.system=all) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:34.215995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7487318-6688-4535-b1bb-5a5b4749e911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:34.217567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7487318-6688-4535-b1bb-5a5b4749e911 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:34.223979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 152/310] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Jun 6 14:07:34 2022] GET /identity/v3/role_assignments?user.id=be276d2d38d244dd989ccfdad02efb27&scope.system=all&role.id=443ba7d155f54b0f85e995b760bd77bb => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:34.555671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d67c9-3ddd-4f3d-b630-4c3ff250ef67 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:34.556233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d67c9-3ddd-4f3d-b630-4c3ff250ef67 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:34.556233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d67c9-3ddd-4f3d-b630-4c3ff250ef67 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:34.558606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 159/311] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:34.564713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee14f5-6805-45db-a5fd-a58158168f49 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:34.565235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee14f5-6805-45db-a5fd-a58158168f49 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:34.565339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee14f5-6805-45db-a5fd-a58158168f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:34.585476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-93ee14f5-6805-45db-a5fd-a58158168f49 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:34.586806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-93ee14f5-6805-45db-a5fd-a58158168f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:34.710629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 153/312] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:34.734938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d67c9-3ddd-4f3d-b630-4c3ff250ef67 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:34.738253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:34.866235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d67c9-3ddd-4f3d-b630-4c3ff250ef67 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:34.905740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11342e-db2b-45eb-9a67-5c973ebf1ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:34.906550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11342e-db2b-45eb-9a67-5c973ebf1ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:34.906832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11342e-db2b-45eb-9a67-5c973ebf1ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:34.907105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11342e-db2b-45eb-9a67-5c973ebf1ed0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:34.919704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11342e-db2b-45eb-9a67-5c973ebf1ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-4d11342e-db2b-45eb-9a67-5c973ebf1ed0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:34.927533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:34.928496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:34.929291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 160/313] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:34.935365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee14f5-6805-45db-a5fd-a58158168f49 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:34.973366 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:35.052382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee14f5-6805-45db-a5fd-a58158168f49 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:35.091728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34078feb-0a04-4f57-b1e4-5bb08b0fcc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:35.092414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34078feb-0a04-4f57-b1e4-5bb08b0fcc15 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:35.092699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34078feb-0a04-4f57-b1e4-5bb08b0fcc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:35.092977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34078feb-0a04-4f57-b1e4-5bb08b0fcc15 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:35.093765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34078feb-0a04-4f57-b1e4-5bb08b0fcc15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:35.094076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34078feb-0a04-4f57-b1e4-5bb08b0fcc15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:35.095486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34078feb-0a04-4f57-b1e4-5bb08b0fcc15 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:35.108892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 154/314] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:35.114231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11342e-db2b-45eb-9a67-5c973ebf1ed0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:35.190505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:35.248748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11342e-db2b-45eb-9a67-5c973ebf1ed0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:35.290465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd9bda-1b6b-487d-bd04-637ccac033c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:35.292146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd9bda-1b6b-487d-bd04-637ccac033c0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:35.292481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd9bda-1b6b-487d-bd04-637ccac033c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:35.292765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd9bda-1b6b-487d-bd04-637ccac033c0 None admin] PATH_INFO: `/v3/users/86fba8a184714aaeb0f28ea7783aae9c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:35.294131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd9bda-1b6b-487d-bd04-637ccac033c0 None admin] RBAC: Authorizing `identity:get_user(user_id=86fba8a184714aaeb0f28ea7783aae9c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:35.295610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd9bda-1b6b-487d-bd04-637ccac033c0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:35.297669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 161/315] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:35 2022] GET /identity/v3/users/86fba8a184714aaeb0f28ea7783aae9c => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:35.308117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34078feb-0a04-4f57-b1e4-5bb08b0fcc15 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:35.505388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34078feb-0a04-4f57-b1e4-5bb08b0fcc15 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:35.540034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:35.565678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39ac61-43e7-419a-9170-c6aee80962ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:35.566858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39ac61-43e7-419a-9170-c6aee80962ff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:35.567368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39ac61-43e7-419a-9170-c6aee80962ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:35.567876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39ac61-43e7-419a-9170-c6aee80962ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:35.691563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39ac61-43e7-419a-9170-c6aee80962ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:35.693473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39ac61-43e7-419a-9170-c6aee80962ff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:35.806779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:35.807194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 155/316] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:35.812385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd9bda-1b6b-487d-bd04-637ccac033c0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:35.926889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd9bda-1b6b-487d-bd04-637ccac033c0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:35.961589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:35.970104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321b20c-41af-4455-80eb-480a6e7962e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:35.971425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321b20c-41af-4455-80eb-480a6e7962e7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:35.971952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321b20c-41af-4455-80eb-480a6e7962e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:35.972429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321b20c-41af-4455-80eb-480a6e7962e7 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:35.974110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321b20c-41af-4455-80eb-480a6e7962e7 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:35.975938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321b20c-41af-4455-80eb-480a6e7962e7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:35.978479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 162/317] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:35 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:35.984916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39ac61-43e7-419a-9170-c6aee80962ff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:36.095235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39ac61-43e7-419a-9170-c6aee80962ff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:36.134716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb561af9-3dcb-427f-8653-1ff32c1ec788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:36.135829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb561af9-3dcb-427f-8653-1ff32c1ec788 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.136324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb561af9-3dcb-427f-8653-1ff32c1ec788 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.136817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb561af9-3dcb-427f-8653-1ff32c1ec788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.137934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb561af9-3dcb-427f-8653-1ff32c1ec788 None admin] RBAC: Adding query filter params (role.id=443ba7d155f54b0f85e995b760bd77bb, scope.project.id=908f423535c84512acd6b17bd7b971ce, user.id=86fba8a184714aaeb0f28ea7783aae9c) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:36.138472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb561af9-3dcb-427f-8653-1ff32c1ec788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:36.140165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb561af9-3dcb-427f-8653-1ff32c1ec788 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:36.147237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 156/318] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:07:35 2022] GET /identity/v3/role_assignments?user.id=86fba8a184714aaeb0f28ea7783aae9c&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=443ba7d155f54b0f85e995b760bd77bb => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:36.214649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e472-339e-4cf1-ae51-f5c5bdefa8a3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.215296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e472-339e-4cf1-ae51-f5c5bdefa8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.215800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e472-339e-4cf1-ae51-f5c5bdefa8a3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.217808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 163/319] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:36.225658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6798f-596f-46ba-b50d-6af3a80267a7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.225991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6798f-596f-46ba-b50d-6af3a80267a7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.226292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6798f-596f-46ba-b50d-6af3a80267a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.245694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9da6798f-596f-46ba-b50d-6af3a80267a7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:36.246936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9da6798f-596f-46ba-b50d-6af3a80267a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:36.314066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653ec9a-c6df-49a4-ac14-14f1d70721d2 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.314735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653ec9a-c6df-49a4-ac14-14f1d70721d2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.315219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653ec9a-c6df-49a4-ac14-14f1d70721d2 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.315506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:36.317263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 164/320] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:36.324660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acb1bc-d46f-4490-91c6-cb1df7024074 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.325266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acb1bc-d46f-4490-91c6-cb1df7024074 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.325826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acb1bc-d46f-4490-91c6-cb1df7024074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.330061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 157/321] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:36.345878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-56acb1bc-d46f-4490-91c6-cb1df7024074 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:36.347491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-56acb1bc-d46f-4490-91c6-cb1df7024074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:36.370687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6798f-596f-46ba-b50d-6af3a80267a7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:36.436820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 165/322] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:36.460562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acb1bc-d46f-4490-91c6-cb1df7024074 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:36.494274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6798f-596f-46ba-b50d-6af3a80267a7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:36.551170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8db543-ac3b-4b99-8154-b1ba655d4ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:36.552293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8db543-ac3b-4b99-8154-b1ba655d4ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.552748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8db543-ac3b-4b99-8154-b1ba655d4ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.553128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8db543-ac3b-4b99-8154-b1ba655d4ab2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.566873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8db543-ac3b-4b99-8154-b1ba655d4ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9f8db543-ac3b-4b99-8154-b1ba655d4ab2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:36.575546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:36.576527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:36.577343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:36.579818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 158/323] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:07:36.589934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8db543-ac3b-4b99-8154-b1ba655d4ab2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:36.628443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acb1bc-d46f-4490-91c6-cb1df7024074 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:36.655984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:36.664172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:36.673507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5bf33-c9c7-4db2-b73c-cd693762c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:36.674383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5bf33-c9c7-4db2-b73c-cd693762c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.674672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5bf33-c9c7-4db2-b73c-cd693762c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.675228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5bf33-c9c7-4db2-b73c-cd693762c6b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.676257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5bf33-c9c7-4db2-b73c-cd693762c6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:36.677868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5bf33-c9c7-4db2-b73c-cd693762c6b4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:36.679862 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 166/324] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:07:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:36.685812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5bf33-c9c7-4db2-b73c-cd693762c6b4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:36.750179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8db543-ac3b-4b99-8154-b1ba655d4ab2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:36.790132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ab3ff-b76d-4913-b27a-b7f3a51f1b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:36.791324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ab3ff-b76d-4913-b27a-b7f3a51f1b0d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.792063 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ab3ff-b76d-4913-b27a-b7f3a51f1b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.792539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ab3ff-b76d-4913-b27a-b7f3a51f1b0d None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.793513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ab3ff-b76d-4913-b27a-b7f3a51f1b0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:36.794036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ab3ff-b76d-4913-b27a-b7f3a51f1b0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:36.795666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ab3ff-b76d-4913-b27a-b7f3a51f1b0d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:36.810538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 159/325] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:36.816063 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ab3ff-b76d-4913-b27a-b7f3a51f1b0d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:36.828128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5bf33-c9c7-4db2-b73c-cd693762c6b4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:36.868279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acd36f-781f-48dd-b6ce-e95fe39638e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:36.869487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acd36f-781f-48dd-b6ce-e95fe39638e5 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.869850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acd36f-781f-48dd-b6ce-e95fe39638e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.870094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acd36f-781f-48dd-b6ce-e95fe39638e5 None admin] PATH_INFO: `/v3/groups` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.870815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acd36f-781f-48dd-b6ce-e95fe39638e5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:36.872294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acd36f-781f-48dd-b6ce-e95fe39638e5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:36.893829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 167/326] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Jun 6 14:07:36 2022] POST /identity/v3/groups => generated 226 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:07:36.944132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ab3ff-b76d-4913-b27a-b7f3a51f1b0d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:36.960570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:36.986663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483771eb-452f-4b73-9e43-1f27c3ba6949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:36.987569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483771eb-452f-4b73-9e43-1f27c3ba6949 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:36.987853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483771eb-452f-4b73-9e43-1f27c3ba6949 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:36.988133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483771eb-452f-4b73-9e43-1f27c3ba6949 None admin] PATH_INFO: `/v3/users/ee1bd4f393c540f9885a88a4f10d2f47` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:36.989486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483771eb-452f-4b73-9e43-1f27c3ba6949 None admin] RBAC: Authorizing `identity:get_user(user_id=ee1bd4f393c540f9885a88a4f10d2f47)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:36.991072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483771eb-452f-4b73-9e43-1f27c3ba6949 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:36.993023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 160/327] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:36 2022] GET /identity/v3/users/ee1bd4f393c540f9885a88a4f10d2f47 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:36.999541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acd36f-781f-48dd-b6ce-e95fe39638e5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:37.063475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:37.116235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acd36f-781f-48dd-b6ce-e95fe39638e5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:37.155312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604eaac-67ae-4f2f-82ba-173bbff1596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:37.155908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604eaac-67ae-4f2f-82ba-173bbff1596d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:37.156191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604eaac-67ae-4f2f-82ba-173bbff1596d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:37.156468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604eaac-67ae-4f2f-82ba-173bbff1596d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:37.157346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604eaac-67ae-4f2f-82ba-173bbff1596d None admin] RBAC: Adding query filter params (role.id=4a06d59562de4d598671a462e3eaf62a, user.id=ee1bd4f393c540f9885a88a4f10d2f47, scope.system=all) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:37.157661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604eaac-67ae-4f2f-82ba-173bbff1596d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:37.159233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604eaac-67ae-4f2f-82ba-173bbff1596d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:37.173861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 168/328] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Jun 6 14:07:36 2022] GET /identity/v3/role_assignments?user.id=ee1bd4f393c540f9885a88a4f10d2f47&scope.system=all&role.id=4a06d59562de4d598671a462e3eaf62a => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:37.199636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b3b2f-f846-4791-a864-2d5bb9bea069 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:37.200129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b3b2f-f846-4791-a864-2d5bb9bea069 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:37.200624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b3b2f-f846-4791-a864-2d5bb9bea069 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:37.203381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 161/329] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:37.209977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18e428-ee38-4636-a31d-614c9705ce45 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:37.210654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18e428-ee38-4636-a31d-614c9705ce45 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:37.211138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18e428-ee38-4636-a31d-614c9705ce45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:37.231986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5c18e428-ee38-4636-a31d-614c9705ce45 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:37.233960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5c18e428-ee38-4636-a31d-614c9705ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:37.328796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 169/330] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:37.352485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b3b2f-f846-4791-a864-2d5bb9bea069 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:37.481284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b3b2f-f846-4791-a864-2d5bb9bea069 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:37.523756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25466e-a4a1-44c2-b355-232678359bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:37.524568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25466e-a4a1-44c2-b355-232678359bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:37.524876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25466e-a4a1-44c2-b355-232678359bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:37.525296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25466e-a4a1-44c2-b355-232678359bc7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:37.537610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25466e-a4a1-44c2-b355-232678359bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9b25466e-a4a1-44c2-b355-232678359bc7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:37.548607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:37.549593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:37.550826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 162/331] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:37.555821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18e428-ee38-4636-a31d-614c9705ce45 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:37.616880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:37.696110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18e428-ee38-4636-a31d-614c9705ce45 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:37.746605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5781edd-c490-4065-9f22-4cda698dda5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:37.747227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5781edd-c490-4065-9f22-4cda698dda5c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:37.747515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5781edd-c490-4065-9f22-4cda698dda5c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:37.747791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5781edd-c490-4065-9f22-4cda698dda5c None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:37.748571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5781edd-c490-4065-9f22-4cda698dda5c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:37.748877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5781edd-c490-4065-9f22-4cda698dda5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:37.750340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5781edd-c490-4065-9f22-4cda698dda5c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:37.761992 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 170/332] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:37.768555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25466e-a4a1-44c2-b355-232678359bc7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:37.830614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:37.902489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25466e-a4a1-44c2-b355-232678359bc7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:37.941902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c9a20-4ce0-44d9-9e44-12d03fe2cfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:37.943179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c9a20-4ce0-44d9-9e44-12d03fe2cfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:37.943663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c9a20-4ce0-44d9-9e44-12d03fe2cfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:37.944137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c9a20-4ce0-44d9-9e44-12d03fe2cfd6 None admin] PATH_INFO: `/v3/users/be276d2d38d244dd989ccfdad02efb27` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:37.945583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c9a20-4ce0-44d9-9e44-12d03fe2cfd6 None admin] RBAC: Authorizing `identity:get_user(user_id=be276d2d38d244dd989ccfdad02efb27)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:37.948004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c9a20-4ce0-44d9-9e44-12d03fe2cfd6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:37.950526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 163/333] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:37 2022] GET /identity/v3/users/be276d2d38d244dd989ccfdad02efb27 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:37.956733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5781edd-c490-4065-9f22-4cda698dda5c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:38.163510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5781edd-c490-4065-9f22-4cda698dda5c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:38.178369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:38.239973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed418c7c-f841-423d-9c00-3bfc78b1a4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:38.241492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed418c7c-f841-423d-9c00-3bfc78b1a4ba None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:38.242009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed418c7c-f841-423d-9c00-3bfc78b1a4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:38.242503 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed418c7c-f841-423d-9c00-3bfc78b1a4ba None admin] PATH_INFO: `/v3/system/users/be276d2d38d244dd989ccfdad02efb27/roles/443ba7d155f54b0f85e995b760bd77bb` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:38.251217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed418c7c-f841-423d-9c00-3bfc78b1a4ba None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=be276d2d38d244dd989ccfdad02efb27, role_id=443ba7d155f54b0f85e995b760bd77bb)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:38.253744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed418c7c-f841-423d-9c00-3bfc78b1a4ba None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:38.272848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 171/334] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Jun 6 14:07:37 2022] PUT /identity/v3/system/users/be276d2d38d244dd989ccfdad02efb27/roles/443ba7d155f54b0f85e995b760bd77bb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:40.288750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e000-c891-435b-996a-e9010f4fb609 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:40.289507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e000-c891-435b-996a-e9010f4fb609 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:40.289633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e000-c891-435b-996a-e9010f4fb609 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:40.292961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 164/335] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:40.299547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a8f50-c472-47e3-93b9-66fa71a4baee None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:40.300292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a8f50-c472-47e3-93b9-66fa71a4baee None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:40.300624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a8f50-c472-47e3-93b9-66fa71a4baee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:40.328557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-573a8f50-c472-47e3-93b9-66fa71a4baee None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:40.329878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-573a8f50-c472-47e3-93b9-66fa71a4baee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:40.431379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 172/336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:40.468166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e000-c891-435b-996a-e9010f4fb609 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:40.752428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e000-c891-435b-996a-e9010f4fb609 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:40.781440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:40.833645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ad660-4555-48c3-a75d-98a51859c771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:40.834388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ad660-4555-48c3-a75d-98a51859c771 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:40.834673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ad660-4555-48c3-a75d-98a51859c771 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:40.834955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ad660-4555-48c3-a75d-98a51859c771 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:40.851877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ad660-4555-48c3-a75d-98a51859c771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-009ad660-4555-48c3-a75d-98a51859c771 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:40.860663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:40.862455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:40.864037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:40.869747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 165/337] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:40.875386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a8f50-c472-47e3-93b9-66fa71a4baee None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:41.136783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:41.143244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a8f50-c472-47e3-93b9-66fa71a4baee None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:41.222064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61760967-6274-4c43-b37a-6ab238ed6c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:41.223476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61760967-6274-4c43-b37a-6ab238ed6c70 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:41.224155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61760967-6274-4c43-b37a-6ab238ed6c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:41.224830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61760967-6274-4c43-b37a-6ab238ed6c70 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:41.226514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61760967-6274-4c43-b37a-6ab238ed6c70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:41.227342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61760967-6274-4c43-b37a-6ab238ed6c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:41.230578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61760967-6274-4c43-b37a-6ab238ed6c70 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:41.252656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 173/338] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:41.261067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ad660-4555-48c3-a75d-98a51859c771 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:41.418675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ad660-4555-48c3-a75d-98a51859c771 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:41.461534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ef11a-cc3f-49f2-ab8a-6c2c411a5528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:41.462257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ef11a-cc3f-49f2-ab8a-6c2c411a5528 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:41.463121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ef11a-cc3f-49f2-ab8a-6c2c411a5528 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:41.463637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ef11a-cc3f-49f2-ab8a-6c2c411a5528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:41.519002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:41.561285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ef11a-cc3f-49f2-ab8a-6c2c411a5528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:41.562969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ef11a-cc3f-49f2-ab8a-6c2c411a5528 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:41.707890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 166/339] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:41.719584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61760967-6274-4c43-b37a-6ab238ed6c70 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:41.973685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61760967-6274-4c43-b37a-6ab238ed6c70 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:42.052145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d8678-f494-427e-b983-8a5ec8805fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:42.054148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d8678-f494-427e-b983-8a5ec8805fcd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.054948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d8678-f494-427e-b983-8a5ec8805fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.055729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d8678-f494-427e-b983-8a5ec8805fcd None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:42.063189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d8678-f494-427e-b983-8a5ec8805fcd None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:42.066467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d8678-f494-427e-b983-8a5ec8805fcd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:42.070258 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 174/340] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:41 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:42.078910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ef11a-cc3f-49f2-ab8a-6c2c411a5528 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:42.111681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9e1fe-172f-4f98-97ac-2b9dba22d1ed None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.112665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9e1fe-172f-4f98-97ac-2b9dba22d1ed None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.113571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9e1fe-172f-4f98-97ac-2b9dba22d1ed None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:42.116853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 175/341] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:42.127600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586377ba-330d-4596-8823-5040fb5cde63 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.128517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586377ba-330d-4596-8823-5040fb5cde63 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.129452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586377ba-330d-4596-8823-5040fb5cde63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:42.185905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-586377ba-330d-4596-8823-5040fb5cde63 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:42.188433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-586377ba-330d-4596-8823-5040fb5cde63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:42.248147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:42.274869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:42.399086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ef11a-cc3f-49f2-ab8a-6c2c411a5528 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:42.408533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 176/342] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:42.410938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b8e59-dfc2-4b96-b055-0f30072030f6 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.411441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b8e59-dfc2-4b96-b055-0f30072030f6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.412004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b8e59-dfc2-4b96-b055-0f30072030f6 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:42.414735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 177/343] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:42.438194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82334f08-3a5e-4eae-a071-9bed3bd7319a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.438683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82334f08-3a5e-4eae-a071-9bed3bd7319a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.439177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82334f08-3a5e-4eae-a071-9bed3bd7319a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:42.470020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0934cce-8230-47cd-ae93-e23b7a64b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:42.470915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0934cce-8230-47cd-ae93-e23b7a64b085 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.471386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0934cce-8230-47cd-ae93-e23b7a64b085 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.471803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0934cce-8230-47cd-ae93-e23b7a64b085 None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:42.485254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0934cce-8230-47cd-ae93-e23b7a64b085 None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:42.487086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0934cce-8230-47cd-ae93-e23b7a64b085 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:42.489428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 167/344] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:42 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:42.492201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-82334f08-3a5e-4eae-a071-9bed3bd7319a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:42.494448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-82334f08-3a5e-4eae-a071-9bed3bd7319a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:42.496205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0934cce-8230-47cd-ae93-e23b7a64b085 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:42.625990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0934cce-8230-47cd-ae93-e23b7a64b085 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:42.674381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cc60a-1819-452c-8011-5bc77d03cba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:42.675356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cc60a-1819-452c-8011-5bc77d03cba6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.675677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cc60a-1819-452c-8011-5bc77d03cba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.676432 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cc60a-1819-452c-8011-5bc77d03cba6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:42.677575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cc60a-1819-452c-8011-5bc77d03cba6 None admin] RBAC: Adding query filter params (group.id=381f93c05f844778af1b9ba657aa3564, role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:42.677914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cc60a-1819-452c-8011-5bc77d03cba6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:42.679517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cc60a-1819-452c-8011-5bc77d03cba6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:42.680867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 178/345] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:42.682350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82334f08-3a5e-4eae-a071-9bed3bd7319a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:42.685617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:42.697301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 168/346] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:42 2022] GET /identity/v3/role_assignments?group.id=381f93c05f844778af1b9ba657aa3564&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=4a06d59562de4d598671a462e3eaf62a => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:42.698968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88db5-ece6-453e-a3c8-93422d916211 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.699438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88db5-ece6-453e-a3c8-93422d916211 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.700536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88db5-ece6-453e-a3c8-93422d916211 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:42.702201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 169/347] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:42.715742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e41ab-0812-45f1-9e41-d061bc09f3b3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.716296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e41ab-0812-45f1-9e41-d061bc09f3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.716718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e41ab-0812-45f1-9e41-d061bc09f3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:42.750758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-207e41ab-0812-45f1-9e41-d061bc09f3b3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:42.752005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-207e41ab-0812-45f1-9e41-d061bc09f3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:42.911084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:42.923376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82334f08-3a5e-4eae-a071-9bed3bd7319a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:42.940349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 170/348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:42.941820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e41ab-0812-45f1-9e41-d061bc09f3b3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:42.992245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8c91-c487-4eb8-8440-0d768577e0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:42.993091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8c91-c487-4eb8-8440-0d768577e0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:42.993629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8c91-c487-4eb8-8440-0d768577e0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:42.994607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8c91-c487-4eb8-8440-0d768577e0a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:43.013127 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a8c91-c487-4eb8-8440-0d768577e0a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-945a8c91-c487-4eb8-8440-0d768577e0a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:43.022014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:43.023030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:43.023872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:43.027029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 179/349] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:43.028284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8c91-c487-4eb8-8440-0d768577e0a0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:43.160913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8c91-c487-4eb8-8440-0d768577e0a0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:43.194191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e41ab-0812-45f1-9e41-d061bc09f3b3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:43.212256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26bfa0-4c7a-476d-8f5d-33546bd5dc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:43.213185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26bfa0-4c7a-476d-8f5d-33546bd5dc59 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:43.213481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26bfa0-4c7a-476d-8f5d-33546bd5dc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:43.213769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26bfa0-4c7a-476d-8f5d-33546bd5dc59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:43.236397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26bfa0-4c7a-476d-8f5d-33546bd5dc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-ff26bfa0-4c7a-476d-8f5d-33546bd5dc59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:43.252880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:43.254866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 180/350] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:43.256523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26bfa0-4c7a-476d-8f5d-33546bd5dc59 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:43.261348 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:43.279960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9d4f8-93b1-4fac-a8a1-84a2ce83352f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:43.282565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9d4f8-93b1-4fac-a8a1-84a2ce83352f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:43.283226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9d4f8-93b1-4fac-a8a1-84a2ce83352f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:43.283920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9d4f8-93b1-4fac-a8a1-84a2ce83352f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:43.290094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:43.322438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9d4f8-93b1-4fac-a8a1-84a2ce83352f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-bbc9d4f8-93b1-4fac-a8a1-84a2ce83352f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:43.338184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:43.339711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:43.341291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:43.351745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 171/351] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:43.355644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9d4f8-93b1-4fac-a8a1-84a2ce83352f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:43.585871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26bfa0-4c7a-476d-8f5d-33546bd5dc59 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:43.668766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9d4f8-93b1-4fac-a8a1-84a2ce83352f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:43.703284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353592b-b1e4-4a25-ae8b-62cab511f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:43.704004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353592b-b1e4-4a25-ae8b-62cab511f2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:43.704284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353592b-b1e4-4a25-ae8b-62cab511f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:43.704565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353592b-b1e4-4a25-ae8b-62cab511f2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:43.705370 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353592b-b1e4-4a25-ae8b-62cab511f2e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:43.705680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353592b-b1e4-4a25-ae8b-62cab511f2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:43.707175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353592b-b1e4-4a25-ae8b-62cab511f2e1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:43.721859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 181/352] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:43.722758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353592b-b1e4-4a25-ae8b-62cab511f2e1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:43.760406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1346701-121c-429c-8943-96e84ca651e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:43.762352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1346701-121c-429c-8943-96e84ca651e2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:43.763307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1346701-121c-429c-8943-96e84ca651e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:43.764234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1346701-121c-429c-8943-96e84ca651e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:43.766325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1346701-121c-429c-8943-96e84ca651e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:43.767206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1346701-121c-429c-8943-96e84ca651e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:43.770612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1346701-121c-429c-8943-96e84ca651e2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:43.792451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 172/353] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:43.794456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1346701-121c-429c-8943-96e84ca651e2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:43.822479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:43.883286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353592b-b1e4-4a25-ae8b-62cab511f2e1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:43.923178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3236402-1b44-4305-9d04-14454d8137ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:43.924766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3236402-1b44-4305-9d04-14454d8137ff None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:43.925221 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3236402-1b44-4305-9d04-14454d8137ff None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:43.925648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3236402-1b44-4305-9d04-14454d8137ff None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:43.926090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:43.929368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3236402-1b44-4305-9d04-14454d8137ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:43.929831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3236402-1b44-4305-9d04-14454d8137ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:43.932374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3236402-1b44-4305-9d04-14454d8137ff None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:43.941254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 182/354] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:43.942191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3236402-1b44-4305-9d04-14454d8137ff None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:44.008694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1346701-121c-429c-8943-96e84ca651e2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:44.050325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf8089-97f0-4165-948b-b7dd6cc758c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:44.051053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf8089-97f0-4165-948b-b7dd6cc758c9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:44.051340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf8089-97f0-4165-948b-b7dd6cc758c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:44.051622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf8089-97f0-4165-948b-b7dd6cc758c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:44.059799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3236402-1b44-4305-9d04-14454d8137ff None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:44.109725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3dcdd-5033-4c68-ba4b-3cbf00df93e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:44.110351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3dcdd-5033-4c68-ba4b-3cbf00df93e5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:44.110639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3dcdd-5033-4c68-ba4b-3cbf00df93e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:44.110923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3dcdd-5033-4c68-ba4b-3cbf00df93e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:44.114434 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:44.160430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf8089-97f0-4165-948b-b7dd6cc758c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:44.164688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf8089-97f0-4165-948b-b7dd6cc758c9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:44.213430 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3dcdd-5033-4c68-ba4b-3cbf00df93e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:44.216622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3dcdd-5033-4c68-ba4b-3cbf00df93e5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:44.247581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:44.347739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 173/355] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:44.349640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0615-b608-4786-b1fe-6384c3bae3f5 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:44.350184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0615-b608-4786-b1fe-6384c3bae3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:44.350682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f0615-b608-4786-b1fe-6384c3bae3f5 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:44.353299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 174/356] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:44.354834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f0615-b608-4786-b1fe-6384c3bae3f5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:44.387125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 183/357] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:44.388977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eb5bf-59bd-4a1f-8fe1-09d60327be27 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:44.389303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eb5bf-59bd-4a1f-8fe1-09d60327be27 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:44.389806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eb5bf-59bd-4a1f-8fe1-09d60327be27 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:44.391319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 184/358] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:44.392585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730eb5bf-59bd-4a1f-8fe1-09d60327be27 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:44.451528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:44.508532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f0615-b608-4786-b1fe-6384c3bae3f5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:44.581061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730eb5bf-59bd-4a1f-8fe1-09d60327be27 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:44.587906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886bd1f-6cfd-4e7f-956e-c9909db32305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:44.587906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886bd1f-6cfd-4e7f-956e-c9909db32305 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:44.587906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886bd1f-6cfd-4e7f-956e-c9909db32305 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:44.587906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886bd1f-6cfd-4e7f-956e-c9909db32305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:44.615841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:44.660133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547930a7-b3e0-484a-a937-331f5b4a1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:44.660918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547930a7-b3e0-484a-a937-331f5b4a1a2d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:44.661206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547930a7-b3e0-484a-a937-331f5b4a1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:44.661485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547930a7-b3e0-484a-a937-331f5b4a1a2d None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:44.662886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547930a7-b3e0-484a-a937-331f5b4a1a2d None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:44.666580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547930a7-b3e0-484a-a937-331f5b4a1a2d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:44.667786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 185/359] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:44 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:44.669350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029713c3-6a38-4d5f-a19c-4f7e5da27e05 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:44.669684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029713c3-6a38-4d5f-a19c-4f7e5da27e05 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:44.669966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029713c3-6a38-4d5f-a19c-4f7e5da27e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:44.689520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886bd1f-6cfd-4e7f-956e-c9909db32305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:44.692041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886bd1f-6cfd-4e7f-956e-c9909db32305 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:44.694644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-029713c3-6a38-4d5f-a19c-4f7e5da27e05 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:44.695941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-029713c3-6a38-4d5f-a19c-4f7e5da27e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:44.794244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 186/360] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:44.796495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029713c3-6a38-4d5f-a19c-4f7e5da27e05 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:44.836026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:44.876751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 175/361] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:44.878501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be597d-f172-452f-938f-e29d1caf3c60 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:44.878857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be597d-f172-452f-938f-e29d1caf3c60 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:44.879142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be597d-f172-452f-938f-e29d1caf3c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:44.899842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-92be597d-f172-452f-938f-e29d1caf3c60 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:44.901397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-92be597d-f172-452f-938f-e29d1caf3c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:44.947126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029713c3-6a38-4d5f-a19c-4f7e5da27e05 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:44.977503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:45.000724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecee64-a53b-4864-9a19-64248e60a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:45.001973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecee64-a53b-4864-9a19-64248e60a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:45.002253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecee64-a53b-4864-9a19-64248e60a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:45.002531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecee64-a53b-4864-9a19-64248e60a1e5 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:45.003859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecee64-a53b-4864-9a19-64248e60a1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:45.010048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecee64-a53b-4864-9a19-64248e60a1e5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:45.010048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 187/362] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:44 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:45.010048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecee64-a53b-4864-9a19-64248e60a1e5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:45.030607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 176/363] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:45.032557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be597d-f172-452f-938f-e29d1caf3c60 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:45.161142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecee64-a53b-4864-9a19-64248e60a1e5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:45.164590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:45.202611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be597d-f172-452f-938f-e29d1caf3c60 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:45.226277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476bad3-06f1-4274-8b84-2ff75b19929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:45.227648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476bad3-06f1-4274-8b84-2ff75b19929d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:45.228261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476bad3-06f1-4274-8b84-2ff75b19929d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:45.228741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476bad3-06f1-4274-8b84-2ff75b19929d None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:45.230743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476bad3-06f1-4274-8b84-2ff75b19929d None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:45.232474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476bad3-06f1-4274-8b84-2ff75b19929d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:45.235720 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 188/364] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:45 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:45.236819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476bad3-06f1-4274-8b84-2ff75b19929d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:45.267889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18945ae-86bd-4547-abcb-08f4426491af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:45.269277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18945ae-86bd-4547-abcb-08f4426491af None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:45.269893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18945ae-86bd-4547-abcb-08f4426491af None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:45.270381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18945ae-86bd-4547-abcb-08f4426491af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:45.288713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18945ae-86bd-4547-abcb-08f4426491af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e18945ae-86bd-4547-abcb-08f4426491af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:45.297381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:45.298463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:45.299297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:45.300444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 177/365] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:45.305506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18945ae-86bd-4547-abcb-08f4426491af None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:45.393311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476bad3-06f1-4274-8b84-2ff75b19929d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:45.395389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:45.447422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480fb71-8f65-4a11-8efd-0ee0b7ddcb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:45.448315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480fb71-8f65-4a11-8efd-0ee0b7ddcb01 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:45.448751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480fb71-8f65-4a11-8efd-0ee0b7ddcb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:45.449328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480fb71-8f65-4a11-8efd-0ee0b7ddcb01 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:45.450881 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480fb71-8f65-4a11-8efd-0ee0b7ddcb01 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:45.452563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480fb71-8f65-4a11-8efd-0ee0b7ddcb01 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:45.455084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 189/366] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:45 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:45.456167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480fb71-8f65-4a11-8efd-0ee0b7ddcb01 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:45.491443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18945ae-86bd-4547-abcb-08f4426491af None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:45.519599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:45.548297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5838ba8-f9bc-4505-a915-6f15b1376427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:45.549287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5838ba8-f9bc-4505-a915-6f15b1376427 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:45.549638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5838ba8-f9bc-4505-a915-6f15b1376427 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:45.550023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5838ba8-f9bc-4505-a915-6f15b1376427 None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:45.552255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5838ba8-f9bc-4505-a915-6f15b1376427 None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:45.553810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5838ba8-f9bc-4505-a915-6f15b1376427 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:45.555828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 178/367] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:45 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:45.561515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5838ba8-f9bc-4505-a915-6f15b1376427 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:45.627724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480fb71-8f65-4a11-8efd-0ee0b7ddcb01 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:45.688414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5838ba8-f9bc-4505-a915-6f15b1376427 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:45.695612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7878590-acff-48fb-8113-3b3fea6b3cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:45.697081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7878590-acff-48fb-8113-3b3fea6b3cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:45.698651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7878590-acff-48fb-8113-3b3fea6b3cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:45.699172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7878590-acff-48fb-8113-3b3fea6b3cd4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:45.700346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:45.720974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7878590-acff-48fb-8113-3b3fea6b3cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:45.733544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf23cf-0faa-4e42-b08f-31cbd713afe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:45.734258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf23cf-0faa-4e42-b08f-31cbd713afe3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:45.734544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf23cf-0faa-4e42-b08f-31cbd713afe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:45.734820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf23cf-0faa-4e42-b08f-31cbd713afe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:45.735681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf23cf-0faa-4e42-b08f-31cbd713afe3 None admin] RBAC: Adding query filter params (group.id=381f93c05f844778af1b9ba657aa3564, role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=908f423535c84512acd6b17bd7b971ce) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:45.735996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf23cf-0faa-4e42-b08f-31cbd713afe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:45.737501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf23cf-0faa-4e42-b08f-31cbd713afe3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d7878590-acff-48fb-8113-3b3fea6b3cd4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:45.737745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:45.738636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 14:07:45.739459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:45.740080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 190/368] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:45.741457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7878590-acff-48fb-8113-3b3fea6b3cd4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:45.755714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 179/369] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:45 2022] GET /identity/v3/role_assignments?group.id=381f93c05f844778af1b9ba657aa3564&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=4a06d59562de4d598671a462e3eaf62a => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:45.755714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf23cf-0faa-4e42-b08f-31cbd713afe3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:45.950948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:45.964650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7878590-acff-48fb-8113-3b3fea6b3cd4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:45.973793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf23cf-0faa-4e42-b08f-31cbd713afe3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:46.028918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409aaa-0b0c-4a82-993f-2316d7ffead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:46.029605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409aaa-0b0c-4a82-993f-2316d7ffead7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:46.029914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409aaa-0b0c-4a82-993f-2316d7ffead7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:46.030195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409aaa-0b0c-4a82-993f-2316d7ffead7 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:46.031298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409aaa-0b0c-4a82-993f-2316d7ffead7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:46.031604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409aaa-0b0c-4a82-993f-2316d7ffead7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:46.033791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409aaa-0b0c-4a82-993f-2316d7ffead7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:46.039888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88647c46-7e9d-4020-9746-b8543e61db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:46.041445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88647c46-7e9d-4020-9746-b8543e61db17 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:46.042288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88647c46-7e9d-4020-9746-b8543e61db17 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:46.042770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88647c46-7e9d-4020-9746-b8543e61db17 None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:46.043802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 191/370] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:46.044675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409aaa-0b0c-4a82-993f-2316d7ffead7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:46.044817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88647c46-7e9d-4020-9746-b8543e61db17 None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:46.047515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88647c46-7e9d-4020-9746-b8543e61db17 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:46.058506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 180/371] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:45 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:46.058506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88647c46-7e9d-4020-9746-b8543e61db17 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:46.140072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:46.198638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409aaa-0b0c-4a82-993f-2316d7ffead7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:46.246081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88647c46-7e9d-4020-9746-b8543e61db17 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:46.260448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50983931-22cd-4af9-8ac8-cf521875fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:46.261891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50983931-22cd-4af9-8ac8-cf521875fd63 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:46.262356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50983931-22cd-4af9-8ac8-cf521875fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:46.262845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50983931-22cd-4af9-8ac8-cf521875fd63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:46.264486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50983931-22cd-4af9-8ac8-cf521875fd63 None admin] RBAC: Adding query filter params (group.id=381f93c05f844778af1b9ba657aa3564, role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=908f423535c84512acd6b17bd7b971ce) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:46.265048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50983931-22cd-4af9-8ac8-cf521875fd63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:46.267806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50983931-22cd-4af9-8ac8-cf521875fd63 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:46.285774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 192/372] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:46 2022] GET /identity/v3/role_assignments?group.id=381f93c05f844778af1b9ba657aa3564&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:46.285774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cb5c-f0a5-4b2a-b2d5-39cb69984d73 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:46.285774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cb5c-f0a5-4b2a-b2d5-39cb69984d73 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:46.285774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cb5c-f0a5-4b2a-b2d5-39cb69984d73 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:46.285774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 193/373] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:46.287179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594cb5c-f0a5-4b2a-b2d5-39cb69984d73 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:46.299202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0401d0-df97-4be9-88b5-c095d72dc869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:46.299882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0401d0-df97-4be9-88b5-c095d72dc869 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:46.300782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0401d0-df97-4be9-88b5-c095d72dc869 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:46.300782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0401d0-df97-4be9-88b5-c095d72dc869 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:46.301189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0401d0-df97-4be9-88b5-c095d72dc869 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:46.301496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0401d0-df97-4be9-88b5-c095d72dc869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:46.303334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0401d0-df97-4be9-88b5-c095d72dc869 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:46.325131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 181/374] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:46.327174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0401d0-df97-4be9-88b5-c095d72dc869 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:46.342608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:46.512287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0401d0-df97-4be9-88b5-c095d72dc869 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:46.515099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:46.518130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594cb5c-f0a5-4b2a-b2d5-39cb69984d73 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:46.572733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd327e3-6879-4895-864b-038afa3d7f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:46.573656 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd327e3-6879-4895-864b-038afa3d7f7c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:46.574245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd327e3-6879-4895-864b-038afa3d7f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:46.574683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd327e3-6879-4895-864b-038afa3d7f7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:46.575964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd327e3-6879-4895-864b-038afa3d7f7c None admin] RBAC: Adding query filter params (group.id=381f93c05f844778af1b9ba657aa3564, role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:46.576428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd327e3-6879-4895-864b-038afa3d7f7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:46.578370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd327e3-6879-4895-864b-038afa3d7f7c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:46.587131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c50189-1cfb-4547-b649-ec89ab4a8103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:46.588932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c50189-1cfb-4547-b649-ec89ab4a8103 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:46.589427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c50189-1cfb-4547-b649-ec89ab4a8103 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:46.589940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 182/375] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:46 2022] GET /identity/v3/role_assignments?group.id=381f93c05f844778af1b9ba657aa3564&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:46.590186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c50189-1cfb-4547-b649-ec89ab4a8103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:46.592241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a569-a614-4627-b5c3-3b74add3f636 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:46.592719 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a569-a614-4627-b5c3-3b74add3f636 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:46.593164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a569-a614-4627-b5c3-3b74add3f636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:46.621103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6708a569-a614-4627-b5c3-3b74add3f636 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:46.625744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6708a569-a614-4627-b5c3-3b74add3f636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:46.771825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c50189-1cfb-4547-b649-ec89ab4a8103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:46.774627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c50189-1cfb-4547-b649-ec89ab4a8103 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:46.791431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 183/376] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:46.792905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708a569-a614-4627-b5c3-3b74add3f636 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:46.823591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:46.974858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:46.985150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708a569-a614-4627-b5c3-3b74add3f636 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:47.029865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 194/377] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 743 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:47.032476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c50189-1cfb-4547-b649-ec89ab4a8103 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:47.040034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54093c-13ae-446b-a7af-cd2a9160fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:47.041580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54093c-13ae-446b-a7af-cd2a9160fc38 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:47.042115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54093c-13ae-446b-a7af-cd2a9160fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:47.042512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54093c-13ae-446b-a7af-cd2a9160fc38 None admin] PATH_INFO: `/v3/users/be276d2d38d244dd989ccfdad02efb27` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:47.046618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54093c-13ae-446b-a7af-cd2a9160fc38 None admin] RBAC: Authorizing `identity:get_user(user_id=be276d2d38d244dd989ccfdad02efb27)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:47.048101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54093c-13ae-446b-a7af-cd2a9160fc38 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:47.049860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 184/378] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:07:46 2022] GET /identity/v3/users/be276d2d38d244dd989ccfdad02efb27 => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:47.050639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54093c-13ae-446b-a7af-cd2a9160fc38 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:47.247373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c50189-1cfb-4547-b649-ec89ab4a8103 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:47.274181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54093c-13ae-446b-a7af-cd2a9160fc38 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:47.298329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:47.322732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14736b-53a6-4888-88a9-de7c2d64367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:47.325111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14736b-53a6-4888-88a9-de7c2d64367c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:47.325915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14736b-53a6-4888-88a9-de7c2d64367c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:47.327104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14736b-53a6-4888-88a9-de7c2d64367c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:47.346848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14736b-53a6-4888-88a9-de7c2d64367c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-cd14736b-53a6-4888-88a9-de7c2d64367c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:47.355947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:47.356969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:47.357798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:47.361102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 195/379] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:47.366673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14736b-53a6-4888-88a9-de7c2d64367c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:47.367060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b6857-5485-431a-b778-202f2c28fe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:47.368625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b6857-5485-431a-b778-202f2c28fe85 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:47.369689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b6857-5485-431a-b778-202f2c28fe85 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:47.370220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b6857-5485-431a-b778-202f2c28fe85 None admin] PATH_INFO: `/v3/projects/258fdfe108b84b6c82c61b2a2990c11d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:47.374687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b6857-5485-431a-b778-202f2c28fe85 None admin] RBAC: Authorizing `identity:get_project(project_id=258fdfe108b84b6c82c61b2a2990c11d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:47.382046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b6857-5485-431a-b778-202f2c28fe85 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:47.384316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 185/380] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:47 2022] GET /identity/v3/projects/258fdfe108b84b6c82c61b2a2990c11d => generated 344 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:47.385458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b6857-5485-431a-b778-202f2c28fe85 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:47.479532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:47.500407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14736b-53a6-4888-88a9-de7c2d64367c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:47.545965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8e366-3feb-438f-ab55-690a2492419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:47.546658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8e366-3feb-438f-ab55-690a2492419d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:47.546943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8e366-3feb-438f-ab55-690a2492419d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:47.547216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8e366-3feb-438f-ab55-690a2492419d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:47.548092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8e366-3feb-438f-ab55-690a2492419d None admin] RBAC: Adding query filter params (role.id=443ba7d155f54b0f85e995b760bd77bb, user.id=be276d2d38d244dd989ccfdad02efb27, scope.system=all) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:47.548465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8e366-3feb-438f-ab55-690a2492419d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:47.550016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8e366-3feb-438f-ab55-690a2492419d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:47.581663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 196/381] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Jun 6 14:07:47 2022] GET /identity/v3/role_assignments?user.id=be276d2d38d244dd989ccfdad02efb27&scope.system=all&role.id=443ba7d155f54b0f85e995b760bd77bb => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:47.582689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8e366-3feb-438f-ab55-690a2492419d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:47.634867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b6857-5485-431a-b778-202f2c28fe85 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:47.722612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1cc26-2f83-4c4a-8fa2-6172f82d06d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:47.724134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1cc26-2f83-4c4a-8fa2-6172f82d06d9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:47.724411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1cc26-2f83-4c4a-8fa2-6172f82d06d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:47.724691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1cc26-2f83-4c4a-8fa2-6172f82d06d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:47.725512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1cc26-2f83-4c4a-8fa2-6172f82d06d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:47.725844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1cc26-2f83-4c4a-8fa2-6172f82d06d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:47.727241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1cc26-2f83-4c4a-8fa2-6172f82d06d9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:47.734264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:47.746007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 186/382] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:47.754835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1cc26-2f83-4c4a-8fa2-6172f82d06d9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:47.858821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8e366-3feb-438f-ab55-690a2492419d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:47.939393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47689f-0d04-45c5-a52f-770f35676916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:47.940280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47689f-0d04-45c5-a52f-770f35676916 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:47.940561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47689f-0d04-45c5-a52f-770f35676916 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:47.940834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47689f-0d04-45c5-a52f-770f35676916 None admin] PATH_INFO: `/v3/groups/e998321c9d7948f891d389130bf1c2f3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:47.958619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47689f-0d04-45c5-a52f-770f35676916 None admin] RBAC: Authorizing `identity:get_group(group_id=e998321c9d7948f891d389130bf1c2f3)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:47.960195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47689f-0d04-45c5-a52f-770f35676916 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:47.962391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 197/383] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:47 2022] GET /identity/v3/groups/e998321c9d7948f891d389130bf1c2f3 => generated 229 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:47.971912 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47689f-0d04-45c5-a52f-770f35676916 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:47.997277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:48.014424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1cc26-2f83-4c4a-8fa2-6172f82d06d9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:48.128862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24910-2037-4def-8523-26391c900906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:48.130267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24910-2037-4def-8523-26391c900906 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:48.130778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24910-2037-4def-8523-26391c900906 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:48.131281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24910-2037-4def-8523-26391c900906 None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:48.133743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf24910-2037-4def-8523-26391c900906 None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:48.136896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf24910-2037-4def-8523-26391c900906 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:48.140163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 187/384] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:47 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:48.153890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24910-2037-4def-8523-26391c900906 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:48.316533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47689f-0d04-45c5-a52f-770f35676916 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:48.346188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:48.406267 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae3a0f-0f9a-4086-9424-5996624f816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:48.406871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae3a0f-0f9a-4086-9424-5996624f816c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:48.407160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae3a0f-0f9a-4086-9424-5996624f816c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:48.407436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae3a0f-0f9a-4086-9424-5996624f816c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:48.408320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae3a0f-0f9a-4086-9424-5996624f816c None admin] RBAC: Adding query filter params (group.id=e998321c9d7948f891d389130bf1c2f3, role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=258fdfe108b84b6c82c61b2a2990c11d) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:48.408636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae3a0f-0f9a-4086-9424-5996624f816c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:48.410224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae3a0f-0f9a-4086-9424-5996624f816c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:48.420477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 198/385] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:47 2022] GET /identity/v3/role_assignments?group.id=e998321c9d7948f891d389130bf1c2f3&scope.project.id=258fdfe108b84b6c82c61b2a2990c11d&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 260 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:48.491308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24910-2037-4def-8523-26391c900906 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:48.582990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1654b7b-d7b2-45a3-9d74-17057c8080ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:48.584342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1654b7b-d7b2-45a3-9d74-17057c8080ff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:48.584781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1654b7b-d7b2-45a3-9d74-17057c8080ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:48.585202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1654b7b-d7b2-45a3-9d74-17057c8080ff None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:48.591227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1654b7b-d7b2-45a3-9d74-17057c8080ff None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:48.593760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1654b7b-d7b2-45a3-9d74-17057c8080ff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:48.597173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 188/386] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:48 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:48.604204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae3a0f-0f9a-4086-9424-5996624f816c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:48.779872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae3a0f-0f9a-4086-9424-5996624f816c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:48.828898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad2575-4ba2-406f-9d2d-c6e877f26b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:48.829901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad2575-4ba2-406f-9d2d-c6e877f26b3a None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:48.830256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad2575-4ba2-406f-9d2d-c6e877f26b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:48.830537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad2575-4ba2-406f-9d2d-c6e877f26b3a None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/groups/381f93c05f844778af1b9ba657aa3564/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:48.832754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad2575-4ba2-406f-9d2d-c6e877f26b3a None admin] RBAC: Authorizing `identity:create_grant(project_id=574e2d880a9446af80a6bbdfa7f0b125, group_id=381f93c05f844778af1b9ba657aa3564, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:48.834282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad2575-4ba2-406f-9d2d-c6e877f26b3a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:48.853850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 199/387] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 6 14:07:48 2022] PUT /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/groups/381f93c05f844778af1b9ba657aa3564/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:48.987384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1ccfe-16df-4eda-bf00-169605cf7362 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:48.987780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1ccfe-16df-4eda-bf00-169605cf7362 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:48.987977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1ccfe-16df-4eda-bf00-169605cf7362 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:48.989637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 189/388] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:48.997040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7977f-80e8-473d-a624-7bdcc158586c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:48.997696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7977f-80e8-473d-a624-7bdcc158586c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:48.998257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7977f-80e8-473d-a624-7bdcc158586c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:49.023659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-43f7977f-80e8-473d-a624-7bdcc158586c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:49.025849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-43f7977f-80e8-473d-a624-7bdcc158586c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:49.061247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:49.125305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 200/389] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:49.153060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1ccfe-16df-4eda-bf00-169605cf7362 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:49.206805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:49.331571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1ccfe-16df-4eda-bf00-169605cf7362 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:49.392227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863b0f4-2b3b-483e-82e1-662f3689f267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:49.393031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863b0f4-2b3b-483e-82e1-662f3689f267 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:49.393318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863b0f4-2b3b-483e-82e1-662f3689f267 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:49.393590 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863b0f4-2b3b-483e-82e1-662f3689f267 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:49.414066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863b0f4-2b3b-483e-82e1-662f3689f267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-6863b0f4-2b3b-483e-82e1-662f3689f267 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:49.423068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:49.424218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:49.425924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 190/390] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:49.430534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7977f-80e8-473d-a624-7bdcc158586c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:49.657055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7977f-80e8-473d-a624-7bdcc158586c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:49.713541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9e3a1-3200-4a4a-ba6d-22eafad99ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:49.714298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9e3a1-3200-4a4a-ba6d-22eafad99ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:49.714539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9e3a1-3200-4a4a-ba6d-22eafad99ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:49.714740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9e3a1-3200-4a4a-ba6d-22eafad99ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:49.715501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9e3a1-3200-4a4a-ba6d-22eafad99ff0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:49.715810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9e3a1-3200-4a4a-ba6d-22eafad99ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:49.717220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9e3a1-3200-4a4a-ba6d-22eafad99ff0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:49.734018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 201/391] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:49.739322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863b0f4-2b3b-483e-82e1-662f3689f267 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:49.787513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63850940-9b7b-4535-9e20-92f7664b64f9 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:49.787805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63850940-9b7b-4535-9e20-92f7664b64f9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:49.788085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63850940-9b7b-4535-9e20-92f7664b64f9 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:49.789769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 202/392] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:49.795612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc723e19-a5a9-4826-bbf1-2510de60a18e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:49.796243 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc723e19-a5a9-4826-bbf1-2510de60a18e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:49.796717 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc723e19-a5a9-4826-bbf1-2510de60a18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:49.847593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-fc723e19-a5a9-4826-bbf1-2510de60a18e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:49.850285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fc723e19-a5a9-4826-bbf1-2510de60a18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:49.889436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:49.912871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:49.932177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863b0f4-2b3b-483e-82e1-662f3689f267 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:50.016029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 203/393] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:50.021800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e08889-ef50-4aae-b34d-df4057dfd532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:50.021800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e08889-ef50-4aae-b34d-df4057dfd532 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:50.021800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e08889-ef50-4aae-b34d-df4057dfd532 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:50.021800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e08889-ef50-4aae-b34d-df4057dfd532 None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:50.022669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e08889-ef50-4aae-b34d-df4057dfd532 None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:50.023998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e08889-ef50-4aae-b34d-df4057dfd532 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:50.034328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 191/394] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:49 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:50.047578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc723e19-a5a9-4826-bbf1-2510de60a18e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:50.117826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e08889-ef50-4aae-b34d-df4057dfd532 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:50.231783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc723e19-a5a9-4826-bbf1-2510de60a18e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:50.279789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e08889-ef50-4aae-b34d-df4057dfd532 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:50.315656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034e914-2ed9-4f9c-b33a-fd2cbebd61fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:50.317572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034e914-2ed9-4f9c-b33a-fd2cbebd61fe None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:50.318235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034e914-2ed9-4f9c-b33a-fd2cbebd61fe None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:50.318960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034e914-2ed9-4f9c-b33a-fd2cbebd61fe None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:50.321400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034e914-2ed9-4f9c-b33a-fd2cbebd61fe None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:50.322024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7714a8-80ed-43c1-89ca-63aa352a3ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:50.323064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7714a8-80ed-43c1-89ca-63aa352a3ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:50.323208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7714a8-80ed-43c1-89ca-63aa352a3ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:50.323529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7714a8-80ed-43c1-89ca-63aa352a3ad4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:50.326093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034e914-2ed9-4f9c-b33a-fd2cbebd61fe None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:50.334159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 204/395] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:50 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:50.335995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7714a8-80ed-43c1-89ca-63aa352a3ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:50.337911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc749a-ddef-42d3-a0e7-efce4ecce4d5 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:50.339682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc749a-ddef-42d3-a0e7-efce4ecce4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:50.343990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc749a-ddef-42d3-a0e7-efce4ecce4d5 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0f7714a8-80ed-43c1-89ca-63aa352a3ad4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:50.345605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:50.347143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 192/396] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:07:50.348251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7714a8-80ed-43c1-89ca-63aa352a3ad4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:50.351900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 205/397] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:50 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:50.360577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc749a-ddef-42d3-a0e7-efce4ecce4d5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:50.374858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:50.384962 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:50.490673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc749a-ddef-42d3-a0e7-efce4ecce4d5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:50.529644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7714a8-80ed-43c1-89ca-63aa352a3ad4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:50.533009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ae3f6-77f8-4519-ba05-3ae71310fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:50.534049 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ae3f6-77f8-4519-ba05-3ae71310fa96 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:50.534555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ae3f6-77f8-4519-ba05-3ae71310fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:50.535031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ae3f6-77f8-4519-ba05-3ae71310fa96 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:50.535985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ae3f6-77f8-4519-ba05-3ae71310fa96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:50.536489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ae3f6-77f8-4519-ba05-3ae71310fa96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:50.538129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ae3f6-77f8-4519-ba05-3ae71310fa96 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:50.548616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 206/398] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:50.550445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab68d0-c038-4ae4-90ea-0e324a24f6e0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:50.551005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab68d0-c038-4ae4-90ea-0e324a24f6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:50.551502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab68d0-c038-4ae4-90ea-0e324a24f6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:50.572873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-17ab68d0-c038-4ae4-90ea-0e324a24f6e0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:50.574493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-17ab68d0-c038-4ae4-90ea-0e324a24f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:50.597469 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3925c52-a63f-42d0-b7eb-406d6997165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:50.599061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3925c52-a63f-42d0-b7eb-406d6997165e None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:50.599552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3925c52-a63f-42d0-b7eb-406d6997165e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:50.600045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3925c52-a63f-42d0-b7eb-406d6997165e None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce/groups/381f93c05f844778af1b9ba657aa3564/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:50.605098 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3925c52-a63f-42d0-b7eb-406d6997165e None admin] RBAC: Authorizing `identity:create_grant(project_id=908f423535c84512acd6b17bd7b971ce, group_id=381f93c05f844778af1b9ba657aa3564, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:50.607898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3925c52-a63f-42d0-b7eb-406d6997165e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:50.637847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 193/399] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 6 14:07:50 2022] PUT /identity/v3/projects/908f423535c84512acd6b17bd7b971ce/groups/381f93c05f844778af1b9ba657aa3564/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:50.639567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3925c52-a63f-42d0-b7eb-406d6997165e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:50.666390 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 207/400] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:50.690845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab68d0-c038-4ae4-90ea-0e324a24f6e0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:50.716503 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:50.835604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:50.917386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3925c52-a63f-42d0-b7eb-406d6997165e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:50.972367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab68d0-c038-4ae4-90ea-0e324a24f6e0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:50.986933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41177c40-647c-4378-a121-879155a4cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:50.988493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41177c40-647c-4378-a121-879155a4cb97 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:50.988997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41177c40-647c-4378-a121-879155a4cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:50.989503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41177c40-647c-4378-a121-879155a4cb97 None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:50.992067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41177c40-647c-4378-a121-879155a4cb97 None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:50.995899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41177c40-647c-4378-a121-879155a4cb97 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:50.999306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 194/401] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:50 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:51.007223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41177c40-647c-4378-a121-879155a4cb97 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:51.018323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af656d1-0adf-495c-a266-f2ef83309c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:51.019164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af656d1-0adf-495c-a266-f2ef83309c97 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:51.019447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af656d1-0adf-495c-a266-f2ef83309c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:51.019723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af656d1-0adf-495c-a266-f2ef83309c97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:51.036566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af656d1-0adf-495c-a266-f2ef83309c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1af656d1-0adf-495c-a266-f2ef83309c97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:51.045473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:51.046567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:51.047397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:51.051702 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 208/402] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:07:51.053168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af656d1-0adf-495c-a266-f2ef83309c97 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:51.213367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41177c40-647c-4378-a121-879155a4cb97 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:51.220349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:51.306657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af656d1-0adf-495c-a266-f2ef83309c97 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:51.357850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c425862-6e8d-44f9-af2f-5c5415aa7cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:51.360450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c425862-6e8d-44f9-af2f-5c5415aa7cae None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:51.361235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c425862-6e8d-44f9-af2f-5c5415aa7cae None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:51.361959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c425862-6e8d-44f9-af2f-5c5415aa7cae None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:51.369750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c425862-6e8d-44f9-af2f-5c5415aa7cae None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:51.371255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832482-123c-4242-81b9-14bd113fa598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:51.372424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832482-123c-4242-81b9-14bd113fa598 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:51.372565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c425862-6e8d-44f9-af2f-5c5415aa7cae None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:51.372923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832482-123c-4242-81b9-14bd113fa598 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:51.373392 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832482-123c-4242-81b9-14bd113fa598 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:51.374847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832482-123c-4242-81b9-14bd113fa598 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:51.375491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832482-123c-4242-81b9-14bd113fa598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:51.382032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832482-123c-4242-81b9-14bd113fa598 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:51.382507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 195/403] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:51 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:51.386896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c425862-6e8d-44f9-af2f-5c5415aa7cae None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:51.397801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 209/404] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:51.404488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832482-123c-4242-81b9-14bd113fa598 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:51.450941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:51.631471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832482-123c-4242-81b9-14bd113fa598 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:51.682320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:51.685031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c425862-6e8d-44f9-af2f-5c5415aa7cae None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:51.709278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa3bb5-aaed-4de9-a441-f60c8ef91902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:51.710204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3bb5-aaed-4de9-a441-f60c8ef91902 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:51.710490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3bb5-aaed-4de9-a441-f60c8ef91902 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:51.710767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3bb5-aaed-4de9-a441-f60c8ef91902 None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:51.714500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa3bb5-aaed-4de9-a441-f60c8ef91902 None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:51.716005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa3bb5-aaed-4de9-a441-f60c8ef91902 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:51.718078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 210/405] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:51 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:51.723813 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa3bb5-aaed-4de9-a441-f60c8ef91902 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:51.758252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043dd58-f1c4-4cac-a78f-0ca5419bacac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:51.759905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043dd58-f1c4-4cac-a78f-0ca5419bacac None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:51.760405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043dd58-f1c4-4cac-a78f-0ca5419bacac None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:51.760859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043dd58-f1c4-4cac-a78f-0ca5419bacac None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce/groups/381f93c05f844778af1b9ba657aa3564/roles/49053ad44c6c4cd1b932fcdb45321ff9` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:51.764493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043dd58-f1c4-4cac-a78f-0ca5419bacac None admin] RBAC: Authorizing `identity:create_grant(project_id=908f423535c84512acd6b17bd7b971ce, group_id=381f93c05f844778af1b9ba657aa3564, role_id=49053ad44c6c4cd1b932fcdb45321ff9)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:51.767499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043dd58-f1c4-4cac-a78f-0ca5419bacac None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:51.805667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 196/406] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 6 14:07:51 2022] PUT /identity/v3/projects/908f423535c84512acd6b17bd7b971ce/groups/381f93c05f844778af1b9ba657aa3564/roles/49053ad44c6c4cd1b932fcdb45321ff9 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:07:51.959796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e90877-c196-4302-9703-eef56f4f16d3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:51.960177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e90877-c196-4302-9703-eef56f4f16d3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:51.964607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e90877-c196-4302-9703-eef56f4f16d3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:51.964607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 197/407] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:51.973749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bacf9-dbc7-4e1b-97b9-bbb9cdd9e752 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:51.973749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bacf9-dbc7-4e1b-97b9-bbb9cdd9e752 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:51.973749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bacf9-dbc7-4e1b-97b9-bbb9cdd9e752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:52.015505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa3bb5-aaed-4de9-a441-f60c8ef91902 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:52.022064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-169bacf9-dbc7-4e1b-97b9-bbb9cdd9e752 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:52.023324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-169bacf9-dbc7-4e1b-97b9-bbb9cdd9e752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:52.125821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c397ef6-e26c-41d4-b427-d1e895896c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:52.125821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397ef6-e26c-41d4-b427-d1e895896c9f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:52.125821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397ef6-e26c-41d4-b427-d1e895896c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:52.125821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397ef6-e26c-41d4-b427-d1e895896c9f None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:52.133736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c397ef6-e26c-41d4-b427-d1e895896c9f None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:52.133736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c397ef6-e26c-41d4-b427-d1e895896c9f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:52.137189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 211/408] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:51 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:52.149748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c397ef6-e26c-41d4-b427-d1e895896c9f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:52.228774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 198/409] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:52.302816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bacf9-dbc7-4e1b-97b9-bbb9cdd9e752 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:52.306108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:52.306275 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:52.362004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c397ef6-e26c-41d4-b427-d1e895896c9f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:52.475347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394e1bd-7b1f-4859-b3ff-675d4711b451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:52.476222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e1bd-7b1f-4859-b3ff-675d4711b451 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:52.476507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e1bd-7b1f-4859-b3ff-675d4711b451 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:52.476781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e1bd-7b1f-4859-b3ff-675d4711b451 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/groups/381f93c05f844778af1b9ba657aa3564/roles/49053ad44c6c4cd1b932fcdb45321ff9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:52.490536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394e1bd-7b1f-4859-b3ff-675d4711b451 None admin] RBAC: Authorizing `identity:create_grant(project_id=574e2d880a9446af80a6bbdfa7f0b125, group_id=381f93c05f844778af1b9ba657aa3564, role_id=49053ad44c6c4cd1b932fcdb45321ff9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:52.492085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394e1bd-7b1f-4859-b3ff-675d4711b451 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:52.538777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 212/410] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 6 14:07:52 2022] PUT /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125/groups/381f93c05f844778af1b9ba657aa3564/roles/49053ad44c6c4cd1b932fcdb45321ff9 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:52.540778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2290fca-230d-48be-a659-8a08d7cdb869 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:52.541493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2290fca-230d-48be-a659-8a08d7cdb869 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:52.543651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bacf9-dbc7-4e1b-97b9-bbb9cdd9e752 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:52.544522 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2290fca-230d-48be-a659-8a08d7cdb869 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:52.560734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 213/411] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:52 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:52.567421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71f0b4-ec42-479b-b009-18df8acc2f88 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:52.568090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71f0b4-ec42-479b-b009-18df8acc2f88 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:52.568622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71f0b4-ec42-479b-b009-18df8acc2f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:52.621743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3b71f0b4-ec42-479b-b009-18df8acc2f88 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:52.623315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3b71f0b4-ec42-479b-b009-18df8acc2f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:52.629958 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71146cbf-afae-4550-8d8f-af49f35da931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:52.631002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71146cbf-afae-4550-8d8f-af49f35da931 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:52.631290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71146cbf-afae-4550-8d8f-af49f35da931 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:52.631563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71146cbf-afae-4550-8d8f-af49f35da931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:52.644610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71146cbf-afae-4550-8d8f-af49f35da931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-71146cbf-afae-4550-8d8f-af49f35da931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:52.658945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:52.659928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:52.661424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 199/412] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:07:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:52.670809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71146cbf-afae-4550-8d8f-af49f35da931 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:52.762564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:52.776007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 214/413] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:52.813392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71146cbf-afae-4550-8d8f-af49f35da931 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:52.824045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71f0b4-ec42-479b-b009-18df8acc2f88 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:52.861486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b88b0d-307c-461f-a2a3-42908eee73cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:52.862324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b88b0d-307c-461f-a2a3-42908eee73cc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:52.862614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b88b0d-307c-461f-a2a3-42908eee73cc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:52.862944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b88b0d-307c-461f-a2a3-42908eee73cc None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:52.863721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b88b0d-307c-461f-a2a3-42908eee73cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:52.864028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b88b0d-307c-461f-a2a3-42908eee73cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:52.865473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b88b0d-307c-461f-a2a3-42908eee73cc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:52.877029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 200/414] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:52.882540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b88b0d-307c-461f-a2a3-42908eee73cc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:52.946226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:53.001445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71f0b4-ec42-479b-b009-18df8acc2f88 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:53.016572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b88b0d-307c-461f-a2a3-42908eee73cc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:53.054419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:53.063091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f15e59-adf2-45c7-9e15-55261e7321e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:53.064620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f15e59-adf2-45c7-9e15-55261e7321e8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:53.065258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f15e59-adf2-45c7-9e15-55261e7321e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:53.065788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f15e59-adf2-45c7-9e15-55261e7321e8 None admin] PATH_INFO: `/v3/groups/e998321c9d7948f891d389130bf1c2f3` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:53.067471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f15e59-adf2-45c7-9e15-55261e7321e8 None admin] RBAC: Authorizing `identity:get_group(group_id=e998321c9d7948f891d389130bf1c2f3)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:53.069201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f15e59-adf2-45c7-9e15-55261e7321e8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:53.071452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c003a80-d2dd-4a79-b28d-a94bd3bd1daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:53.072547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 201/415] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:52 2022] GET /identity/v3/groups/e998321c9d7948f891d389130bf1c2f3 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:53.073079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c003a80-d2dd-4a79-b28d-a94bd3bd1daf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:53.073368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c003a80-d2dd-4a79-b28d-a94bd3bd1daf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:53.073637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c003a80-d2dd-4a79-b28d-a94bd3bd1daf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:53.077333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f15e59-adf2-45c7-9e15-55261e7321e8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:53.095685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c003a80-d2dd-4a79-b28d-a94bd3bd1daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7c003a80-d2dd-4a79-b28d-a94bd3bd1daf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:53.115719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:53.116704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:53.117512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 215/416] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:53.126882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c003a80-d2dd-4a79-b28d-a94bd3bd1daf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:53.199363 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f15e59-adf2-45c7-9e15-55261e7321e8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:53.240078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d6e39-6bf1-43e0-93ef-8cb53f918770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:53.241471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d6e39-6bf1-43e0-93ef-8cb53f918770 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:53.242173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d6e39-6bf1-43e0-93ef-8cb53f918770 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:53.242717 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d6e39-6bf1-43e0-93ef-8cb53f918770 None admin] PATH_INFO: `/v3/projects/258fdfe108b84b6c82c61b2a2990c11d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:53.245054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d6e39-6bf1-43e0-93ef-8cb53f918770 None admin] RBAC: Authorizing `identity:get_project(project_id=258fdfe108b84b6c82c61b2a2990c11d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:53.246941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d6e39-6bf1-43e0-93ef-8cb53f918770 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:53.249518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 202/417] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:53 2022] GET /identity/v3/projects/258fdfe108b84b6c82c61b2a2990c11d => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:53.254991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d6e39-6bf1-43e0-93ef-8cb53f918770 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:53.338374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:53.391451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d6e39-6bf1-43e0-93ef-8cb53f918770 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:53.475844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c003a80-d2dd-4a79-b28d-a94bd3bd1daf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:53.514489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2650bc9-759f-4eaa-9349-36cdd7f3804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:53.515405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2650bc9-759f-4eaa-9349-36cdd7f3804f None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:53.515719 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2650bc9-759f-4eaa-9349-36cdd7f3804f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:53.515963 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2650bc9-759f-4eaa-9349-36cdd7f3804f None admin] PATH_INFO: `/v3/projects/258fdfe108b84b6c82c61b2a2990c11d/groups/e998321c9d7948f891d389130bf1c2f3/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:53.517346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a59325-986b-4585-aa2e-d391b4972f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:53.518454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a59325-986b-4585-aa2e-d391b4972f69 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:53.518964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a59325-986b-4585-aa2e-d391b4972f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:53.519464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a59325-986b-4585-aa2e-d391b4972f69 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:53.520432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a59325-986b-4585-aa2e-d391b4972f69 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:53.520964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a59325-986b-4585-aa2e-d391b4972f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:53.522575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2650bc9-759f-4eaa-9349-36cdd7f3804f None admin] RBAC: Authorizing `identity:create_grant(project_id=258fdfe108b84b6c82c61b2a2990c11d, group_id=e998321c9d7948f891d389130bf1c2f3, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:53.523065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a59325-986b-4585-aa2e-d391b4972f69 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:53.524036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2650bc9-759f-4eaa-9349-36cdd7f3804f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:53.530160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:53.535132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 216/418] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:53.541311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a59325-986b-4585-aa2e-d391b4972f69 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:53.573821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 203/419] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 6 14:07:53 2022] PUT /identity/v3/projects/258fdfe108b84b6c82c61b2a2990c11d/groups/e998321c9d7948f891d389130bf1c2f3/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:07:53.637522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55c6af-6dac-4ea5-9609-8e29f7cd053d None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:53.637841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55c6af-6dac-4ea5-9609-8e29f7cd053d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:53.638053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55c6af-6dac-4ea5-9609-8e29f7cd053d None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:53.639608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 204/420] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:53.652104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac5c6b-88ea-45dc-ada7-f2777febd32e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:53.652413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac5c6b-88ea-45dc-ada7-f2777febd32e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:53.652677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac5c6b-88ea-45dc-ada7-f2777febd32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:53.672974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a59325-986b-4585-aa2e-d391b4972f69 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:53.690877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4fac5c6b-88ea-45dc-ada7-f2777febd32e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:53.692097 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4fac5c6b-88ea-45dc-ada7-f2777febd32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:53.724753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37d20d-ee8c-4306-ae83-30cf9842adfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:53.726347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37d20d-ee8c-4306-ae83-30cf9842adfa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:53.727039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37d20d-ee8c-4306-ae83-30cf9842adfa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:53.728726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37d20d-ee8c-4306-ae83-30cf9842adfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:53.874147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:53.879650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37d20d-ee8c-4306-ae83-30cf9842adfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:53.883218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37d20d-ee8c-4306-ae83-30cf9842adfa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:53.903898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 205/421] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:53.930518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac5c6b-88ea-45dc-ada7-f2777febd32e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:54.009516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:54.070883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac5c6b-88ea-45dc-ada7-f2777febd32e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:54.119864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 217/422] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:54.125972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37d20d-ee8c-4306-ae83-30cf9842adfa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:54.180030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364cf29-2126-431c-9441-2b9d1327d0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:54.182570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364cf29-2126-431c-9441-2b9d1327d0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:54.183342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364cf29-2126-431c-9441-2b9d1327d0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:54.184020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364cf29-2126-431c-9441-2b9d1327d0c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:54.218919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364cf29-2126-431c-9441-2b9d1327d0c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1364cf29-2126-431c-9441-2b9d1327d0c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:54.235742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:54.237522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:54.240813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 206/423] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:54.248025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364cf29-2126-431c-9441-2b9d1327d0c4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:54.359988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:54.420774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37d20d-ee8c-4306-ae83-30cf9842adfa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:54.457318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364cf29-2126-431c-9441-2b9d1327d0c4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:54.488592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:54.544067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3c510-ac11-4c1b-86f4-e75bae1b077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:54.544960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3c510-ac11-4c1b-86f4-e75bae1b077d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:54.545269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3c510-ac11-4c1b-86f4-e75bae1b077d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:54.545556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3c510-ac11-4c1b-86f4-e75bae1b077d None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:54.549060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3c510-ac11-4c1b-86f4-e75bae1b077d None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:54.551891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3c510-ac11-4c1b-86f4-e75bae1b077d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:54.559594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7452b0-5654-4c0d-a37d-1307a2c95729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:54.560776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7452b0-5654-4c0d-a37d-1307a2c95729 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:54.561292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7452b0-5654-4c0d-a37d-1307a2c95729 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:54.561833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7452b0-5654-4c0d-a37d-1307a2c95729 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:54.563341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7452b0-5654-4c0d-a37d-1307a2c95729 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:54.563687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 218/424] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:54 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:54.563846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7452b0-5654-4c0d-a37d-1307a2c95729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:54.566835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7452b0-5654-4c0d-a37d-1307a2c95729 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:54.569496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3c510-ac11-4c1b-86f4-e75bae1b077d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:54.584414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 207/425] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:07:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:54.593468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7452b0-5654-4c0d-a37d-1307a2c95729 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:54.804704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7452b0-5654-4c0d-a37d-1307a2c95729 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:54.893119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dbf56-b999-4822-a7ea-87592b17b91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:54.893915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dbf56-b999-4822-a7ea-87592b17b91d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:54.894203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dbf56-b999-4822-a7ea-87592b17b91d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:54.894485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dbf56-b999-4822-a7ea-87592b17b91d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:54.959113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:54.967020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3c510-ac11-4c1b-86f4-e75bae1b077d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:54.997830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dbf56-b999-4822-a7ea-87592b17b91d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:54.999417 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dbf56-b999-4822-a7ea-87592b17b91d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:55.067740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40a3bd-8291-46a5-94d8-239034fda148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:55.068529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40a3bd-8291-46a5-94d8-239034fda148 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:55.068811 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40a3bd-8291-46a5-94d8-239034fda148 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:55.069089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40a3bd-8291-46a5-94d8-239034fda148 None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:55.074642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40a3bd-8291-46a5-94d8-239034fda148 None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:55.076210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40a3bd-8291-46a5-94d8-239034fda148 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:55.078016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 219/426] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:54 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 510 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:55.090702 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40a3bd-8291-46a5-94d8-239034fda148 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:55.138629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:55.255940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 208/427] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:55.262721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dbf56-b999-4822-a7ea-87592b17b91d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:55.431765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:55.436565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40a3bd-8291-46a5-94d8-239034fda148 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:55.525510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dbf56-b999-4822-a7ea-87592b17b91d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:55.567824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba75fb-3a06-45e4-b536-10b4a540f2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:55.568937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba75fb-3a06-45e4-b536-10b4a540f2bb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:55.569683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba75fb-3a06-45e4-b536-10b4a540f2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:55.570205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba75fb-3a06-45e4-b536-10b4a540f2bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:55.572131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba75fb-3a06-45e4-b536-10b4a540f2bb None admin] RBAC: Adding query filter params (group.id=381f93c05f844778af1b9ba657aa3564, role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:55.572483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba75fb-3a06-45e4-b536-10b4a540f2bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:55.575260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba75fb-3a06-45e4-b536-10b4a540f2bb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:55.609249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 220/428] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:55 2022] GET /identity/v3/role_assignments?group.id=381f93c05f844778af1b9ba657aa3564&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=4a06d59562de4d598671a462e3eaf62a => generated 611 bytes in 519 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:55.637564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43927e76-568d-4f7a-8000-1522a7c88ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:55.639003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43927e76-568d-4f7a-8000-1522a7c88ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:55.639462 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43927e76-568d-4f7a-8000-1522a7c88ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:55.639945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43927e76-568d-4f7a-8000-1522a7c88ff6 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:55.642125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43927e76-568d-4f7a-8000-1522a7c88ff6 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:55.644593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43927e76-568d-4f7a-8000-1522a7c88ff6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:55.648374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 209/429] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:55 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:55.654958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba75fb-3a06-45e4-b536-10b4a540f2bb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:55.839215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5193fa-bdd1-4822-8215-fe6f5087bf3b None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:55.839722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5193fa-bdd1-4822-8215-fe6f5087bf3b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:55.840137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5193fa-bdd1-4822-8215-fe6f5087bf3b None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:55.843065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 210/430] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:07:55.850508 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25a68e-ab3d-4d91-9f71-33f674895340 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:55.850963 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25a68e-ab3d-4d91-9f71-33f674895340 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:55.851430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25a68e-ab3d-4d91-9f71-33f674895340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:55.882587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:55.903089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5f25a68e-ab3d-4d91-9f71-33f674895340 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:55.905672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5f25a68e-ab3d-4d91-9f71-33f674895340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:55.949226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba75fb-3a06-45e4-b536-10b4a540f2bb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:56.010488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10441856-819e-45cc-b151-a90ed233d93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:56.011444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441856-819e-45cc-b151-a90ed233d93f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:56.011613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441856-819e-45cc-b151-a90ed233d93f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:56.011934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441856-819e-45cc-b151-a90ed233d93f None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:56.013351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10441856-819e-45cc-b151-a90ed233d93f None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:56.014904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10441856-819e-45cc-b151-a90ed233d93f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:56.016928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 221/431] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:55 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:56.018267 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284de7a3-df92-4ac5-ad5e-4096d5630484 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:56.018553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284de7a3-df92-4ac5-ad5e-4096d5630484 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:56.018823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284de7a3-df92-4ac5-ad5e-4096d5630484 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:56.028768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 222/432] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:56.031003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25e188-61c0-44c5-acb1-c2a80fe11f7e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:56.031283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25e188-61c0-44c5-acb1-c2a80fe11f7e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:56.031562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25e188-61c0-44c5-acb1-c2a80fe11f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:56.044056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 211/433] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:56.045142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25a68e-ab3d-4d91-9f71-33f674895340 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:56.058679 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0e25e188-61c0-44c5-acb1-c2a80fe11f7e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:56.060453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0e25e188-61c0-44c5-acb1-c2a80fe11f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:56.172760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:56.208791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 223/434] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:07:56.210011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25e188-61c0-44c5-acb1-c2a80fe11f7e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:56.235070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25a68e-ab3d-4d91-9f71-33f674895340 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:56.300792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca92892-74ca-45f1-8b4d-507c36300452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:56.301471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92892-74ca-45f1-8b4d-507c36300452 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:56.301784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92892-74ca-45f1-8b4d-507c36300452 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:56.302051 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92892-74ca-45f1-8b4d-507c36300452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:56.302923 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca92892-74ca-45f1-8b4d-507c36300452 None admin] RBAC: Adding query filter params (group.id=381f93c05f844778af1b9ba657aa3564, role.id=4a06d59562de4d598671a462e3eaf62a, scope.project.id=908f423535c84512acd6b17bd7b971ce) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:56.303239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca92892-74ca-45f1-8b4d-507c36300452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:56.304721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca92892-74ca-45f1-8b4d-507c36300452 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:56.316704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 212/435] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:56 2022] GET /identity/v3/role_assignments?group.id=381f93c05f844778af1b9ba657aa3564&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=4a06d59562de4d598671a462e3eaf62a => generated 611 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:56.316704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca92892-74ca-45f1-8b4d-507c36300452 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:56.351911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:56.384605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25e188-61c0-44c5-acb1-c2a80fe11f7e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:56.426740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0dd8f-82d5-4b04-b052-5440b68161a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:56.427478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0dd8f-82d5-4b04-b052-5440b68161a9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:56.427767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0dd8f-82d5-4b04-b052-5440b68161a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:56.428049 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0dd8f-82d5-4b04-b052-5440b68161a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:56.442384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0dd8f-82d5-4b04-b052-5440b68161a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-93a0dd8f-82d5-4b04-b052-5440b68161a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:56.452731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:56.454475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:56.455974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 224/436] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:56.466661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0dd8f-82d5-4b04-b052-5440b68161a9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:56.511809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca92892-74ca-45f1-8b4d-507c36300452 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:56.563272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead5c7c-4ebc-4344-95c5-c3b0a528404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:56.564753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead5c7c-4ebc-4344-95c5-c3b0a528404e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:56.565043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead5c7c-4ebc-4344-95c5-c3b0a528404e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:56.565299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead5c7c-4ebc-4344-95c5-c3b0a528404e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:56.568855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:56.584708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead5c7c-4ebc-4344-95c5-c3b0a528404e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-dead5c7c-4ebc-4344-95c5-c3b0a528404e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:56.602110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:56.605036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:07:56.606567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:07:56.610970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 213/437] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:07:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:56.620827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead5c7c-4ebc-4344-95c5-c3b0a528404e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:56.632319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0dd8f-82d5-4b04-b052-5440b68161a9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:56.686550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55737d-19a2-45b1-a532-5aa2d35feb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:56.687165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55737d-19a2-45b1-a532-5aa2d35feb98 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:56.687442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55737d-19a2-45b1-a532-5aa2d35feb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:56.687721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55737d-19a2-45b1-a532-5aa2d35feb98 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:56.688508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55737d-19a2-45b1-a532-5aa2d35feb98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:56.688815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55737d-19a2-45b1-a532-5aa2d35feb98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:56.691097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55737d-19a2-45b1-a532-5aa2d35feb98 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:56.698020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:56.706399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 225/438] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:56.714561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55737d-19a2-45b1-a532-5aa2d35feb98 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:56.801833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead5c7c-4ebc-4344-95c5-c3b0a528404e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:56.840545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccb157-ee35-4689-a3d1-630fef4f2b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:56.841699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccb157-ee35-4689-a3d1-630fef4f2b1c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:56.842241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccb157-ee35-4689-a3d1-630fef4f2b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:56.842760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccb157-ee35-4689-a3d1-630fef4f2b1c None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:56.843524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccb157-ee35-4689-a3d1-630fef4f2b1c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:56.843833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccb157-ee35-4689-a3d1-630fef4f2b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:56.845243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccb157-ee35-4689-a3d1-630fef4f2b1c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:56.855698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 214/439] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:07:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:56.861262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccb157-ee35-4689-a3d1-630fef4f2b1c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:56.889521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55737d-19a2-45b1-a532-5aa2d35feb98 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:56.927539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:56.945823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec9b4c-4199-4308-9ce3-7d806bf0d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:56.946395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec9b4c-4199-4308-9ce3-7d806bf0d0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:56.946849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec9b4c-4199-4308-9ce3-7d806bf0d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:56.946992 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec9b4c-4199-4308-9ce3-7d806bf0d0d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.000680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccb157-ee35-4689-a3d1-630fef4f2b1c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:57.024435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:57.028160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec9b4c-4199-4308-9ce3-7d806bf0d0d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:57.029777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec9b4c-4199-4308-9ce3-7d806bf0d0d0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:57.046605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1c283-e155-4eb9-9a72-d7df3aee1f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:57.047262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1c283-e155-4eb9-9a72-d7df3aee1f29 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:57.047685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1c283-e155-4eb9-9a72-d7df3aee1f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:57.047831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1c283-e155-4eb9-9a72-d7df3aee1f29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.137115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1c283-e155-4eb9-9a72-d7df3aee1f29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:57.138947 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1c283-e155-4eb9-9a72-d7df3aee1f29 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:57.153745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 226/440] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 14:07:57.159378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec9b4c-4199-4308-9ce3-7d806bf0d0d0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:57.216078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:57.267651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 215/441] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 14:07:57.269011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3227a7-eac1-42b0-95cc-6f5d85198b1d None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:57.269306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3227a7-eac1-42b0-95cc-6f5d85198b1d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:57.269557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3227a7-eac1-42b0-95cc-6f5d85198b1d None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.271122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 216/442] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:07:57.276429 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3227a7-eac1-42b0-95cc-6f5d85198b1d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:57.285323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:57.289153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec9b4c-4199-4308-9ce3-7d806bf0d0d0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:57.343078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd884e8-e8d7-4ea2-9016-3948754148e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:57.344450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd884e8-e8d7-4ea2-9016-3948754148e1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:57.344905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd884e8-e8d7-4ea2-9016-3948754148e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:57.345367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd884e8-e8d7-4ea2-9016-3948754148e1 None admin] PATH_INFO: `/v3/projects/574e2d880a9446af80a6bbdfa7f0b125` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.347379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd884e8-e8d7-4ea2-9016-3948754148e1 None admin] RBAC: Authorizing `identity:get_project(project_id=574e2d880a9446af80a6bbdfa7f0b125)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:57.349732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd884e8-e8d7-4ea2-9016-3948754148e1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:57.352655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 227/443] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:57 2022] GET /identity/v3/projects/574e2d880a9446af80a6bbdfa7f0b125 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:57.355499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1826691-4abf-4c97-a202-9c25b1d02775 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:57.355988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1826691-4abf-4c97-a202-9c25b1d02775 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:57.356471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1826691-4abf-4c97-a202-9c25b1d02775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.376017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a1826691-4abf-4c97-a202-9c25b1d02775 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:07:57.377226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a1826691-4abf-4c97-a202-9c25b1d02775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:07:57.409465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3227a7-eac1-42b0-95cc-6f5d85198b1d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:57.446289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1f7ae-a9d0-4e47-adf4-d03baea9d847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:57.447837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1f7ae-a9d0-4e47-adf4-d03baea9d847 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:57.448158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1f7ae-a9d0-4e47-adf4-d03baea9d847 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:57.448433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1f7ae-a9d0-4e47-adf4-d03baea9d847 None admin] PATH_INFO: `/v3/projects/908f423535c84512acd6b17bd7b971ce` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.449676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1f7ae-a9d0-4e47-adf4-d03baea9d847 None admin] RBAC: Authorizing `identity:get_project(project_id=908f423535c84512acd6b17bd7b971ce)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:57.450489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 228/444] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:07:57.451185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1f7ae-a9d0-4e47-adf4-d03baea9d847 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:57.451336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1826691-4abf-4c97-a202-9c25b1d02775 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:57.459815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 217/445] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:57 2022] GET /identity/v3/projects/908f423535c84512acd6b17bd7b971ce => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:57.460893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1f7ae-a9d0-4e47-adf4-d03baea9d847 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:57.501939 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:57.564550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:57.568734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1826691-4abf-4c97-a202-9c25b1d02775 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:57.586045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1f7ae-a9d0-4e47-adf4-d03baea9d847 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:57.612984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100e17e-4c7c-456e-9797-c3103e76ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:57.613891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100e17e-4c7c-456e-9797-c3103e76ec4c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:57.614171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100e17e-4c7c-456e-9797-c3103e76ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:57.614447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100e17e-4c7c-456e-9797-c3103e76ec4c None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.615693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100e17e-4c7c-456e-9797-c3103e76ec4c None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:57.617155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100e17e-4c7c-456e-9797-c3103e76ec4c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:57.619331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 229/446] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:57 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:57.620248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100e17e-4c7c-456e-9797-c3103e76ec4c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:57.629854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03468532-1436-4834-af84-2236d43170b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:57.630814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468532-1436-4834-af84-2236d43170b2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:57.631156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468532-1436-4834-af84-2236d43170b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:57.631487 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468532-1436-4834-af84-2236d43170b2 None admin] PATH_INFO: `/v3/groups/381f93c05f844778af1b9ba657aa3564` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.633028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03468532-1436-4834-af84-2236d43170b2 None admin] RBAC: Authorizing `identity:get_group(group_id=381f93c05f844778af1b9ba657aa3564)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:57.635132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03468532-1436-4834-af84-2236d43170b2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:57.641914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 218/447] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:57 2022] GET /identity/v3/groups/381f93c05f844778af1b9ba657aa3564 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:57.641914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03468532-1436-4834-af84-2236d43170b2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:57.751319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100e17e-4c7c-456e-9797-c3103e76ec4c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:57.768273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:57.779837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03468532-1436-4834-af84-2236d43170b2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:57.805320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae9139-c9b5-4b30-937f-cd5801a7f477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:57.806793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae9139-c9b5-4b30-937f-cd5801a7f477 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:57.807895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae9139-c9b5-4b30-937f-cd5801a7f477 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:57.808232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae9139-c9b5-4b30-937f-cd5801a7f477 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.820677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae9139-c9b5-4b30-937f-cd5801a7f477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-41ae9139-c9b5-4b30-937f-cd5801a7f477 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:07:57.829842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:07:57.831350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 230/448] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:07:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:07:57.833369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae9139-c9b5-4b30-937f-cd5801a7f477 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:57.842412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f126e8-768b-44c3-9d78-c29ba8194f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:57.843530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f126e8-768b-44c3-9d78-c29ba8194f0b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:57.844038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f126e8-768b-44c3-9d78-c29ba8194f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:57.844529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f126e8-768b-44c3-9d78-c29ba8194f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:57.846244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f126e8-768b-44c3-9d78-c29ba8194f0b None admin] RBAC: Adding query filter params (group.id=381f93c05f844778af1b9ba657aa3564, role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=574e2d880a9446af80a6bbdfa7f0b125) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:57.846808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f126e8-768b-44c3-9d78-c29ba8194f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:57.849610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f126e8-768b-44c3-9d78-c29ba8194f0b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:57.861193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 219/449] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:57 2022] GET /identity/v3/role_assignments?group.id=381f93c05f844778af1b9ba657aa3564&scope.project.id=574e2d880a9446af80a6bbdfa7f0b125&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:57.862554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f126e8-768b-44c3-9d78-c29ba8194f0b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:57.870738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:58.019457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae9139-c9b5-4b30-937f-cd5801a7f477 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:58.053587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f126e8-768b-44c3-9d78-c29ba8194f0b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:58.090548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77249507-ca1a-43be-a2d2-08b9bb217d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:58.091991 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77249507-ca1a-43be-a2d2-08b9bb217d8f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:58.092603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77249507-ca1a-43be-a2d2-08b9bb217d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:58.092977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77249507-ca1a-43be-a2d2-08b9bb217d8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:58.094737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77249507-ca1a-43be-a2d2-08b9bb217d8f None admin] RBAC: Adding query filter params (group.id=381f93c05f844778af1b9ba657aa3564, role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=908f423535c84512acd6b17bd7b971ce) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:58.095299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77249507-ca1a-43be-a2d2-08b9bb217d8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:58.098423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77249507-ca1a-43be-a2d2-08b9bb217d8f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:58.116301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 231/450] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:57 2022] GET /identity/v3/role_assignments?group.id=381f93c05f844778af1b9ba657aa3564&scope.project.id=908f423535c84512acd6b17bd7b971ce&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 611 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:58.116510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e069a24-6002-4961-a27d-7b3f6accf819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:58.116510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e069a24-6002-4961-a27d-7b3f6accf819 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:58.116510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e069a24-6002-4961-a27d-7b3f6accf819 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:58.116510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e069a24-6002-4961-a27d-7b3f6accf819 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:58.116816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e069a24-6002-4961-a27d-7b3f6accf819 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:58.116816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e069a24-6002-4961-a27d-7b3f6accf819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:58.116816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e069a24-6002-4961-a27d-7b3f6accf819 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:58.126569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 220/451] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:07:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:58.133967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77249507-ca1a-43be-a2d2-08b9bb217d8f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:58.268604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:58.285805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77249507-ca1a-43be-a2d2-08b9bb217d8f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:58.335494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd9a0c-4f97-4373-8b4a-1dcb64da174f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:58.337133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd9a0c-4f97-4373-8b4a-1dcb64da174f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:58.337882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd9a0c-4f97-4373-8b4a-1dcb64da174f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:58.338345 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd9a0c-4f97-4373-8b4a-1dcb64da174f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:58.435533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd9a0c-4f97-4373-8b4a-1dcb64da174f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:58.439473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd9a0c-4f97-4373-8b4a-1dcb64da174f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:58.594562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:58.610734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 232/452] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:07:58.618423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e069a24-6002-4961-a27d-7b3f6accf819 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:58.652449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:58.789838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e069a24-6002-4961-a27d-7b3f6accf819 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:58.848250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c4abd-c371-48de-a5f4-6e51253a5af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:58.849657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c4abd-c371-48de-a5f4-6e51253a5af3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:58.850191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c4abd-c371-48de-a5f4-6e51253a5af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:58.850690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c4abd-c371-48de-a5f4-6e51253a5af3 None admin] PATH_INFO: `/v3/projects/258fdfe108b84b6c82c61b2a2990c11d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:58.852687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c4abd-c371-48de-a5f4-6e51253a5af3 None admin] RBAC: Authorizing `identity:get_project(project_id=258fdfe108b84b6c82c61b2a2990c11d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:58.856356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c4abd-c371-48de-a5f4-6e51253a5af3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:58.859813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 221/453] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:07:58 2022] GET /identity/v3/projects/258fdfe108b84b6c82c61b2a2990c11d => generated 344 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:07:58.865593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd9a0c-4f97-4373-8b4a-1dcb64da174f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:58.982418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd9a0c-4f97-4373-8b4a-1dcb64da174f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:59.152032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccf575-5b31-44b5-bd51-6385d7c82f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:59.152838 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf575-5b31-44b5-bd51-6385d7c82f07 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:59.153115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf575-5b31-44b5-bd51-6385d7c82f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:59.153393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccf575-5b31-44b5-bd51-6385d7c82f07 None admin] PATH_INFO: `/v3/groups/e998321c9d7948f891d389130bf1c2f3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:59.154674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccf575-5b31-44b5-bd51-6385d7c82f07 None admin] RBAC: Authorizing `identity:get_group(group_id=e998321c9d7948f891d389130bf1c2f3)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:59.156145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccf575-5b31-44b5-bd51-6385d7c82f07 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:59.157840 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 233/454] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 6 14:07:58 2022] GET /identity/v3/groups/e998321c9d7948f891d389130bf1c2f3 => generated 229 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:07:59.164095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c4abd-c371-48de-a5f4-6e51253a5af3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:07:59.262104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:07:59.276994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c4abd-c371-48de-a5f4-6e51253a5af3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:07:59.313946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20092132-b4b6-4d45-aff6-ed570d30fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:07:59.314611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20092132-b4b6-4d45-aff6-ed570d30fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:07:59.314898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20092132-b4b6-4d45-aff6-ed570d30fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:07:59.315178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20092132-b4b6-4d45-aff6-ed570d30fcb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:07:59.316019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20092132-b4b6-4d45-aff6-ed570d30fcb7 None admin] RBAC: Adding query filter params (group.id=e998321c9d7948f891d389130bf1c2f3, role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=258fdfe108b84b6c82c61b2a2990c11d) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:07:59.316331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20092132-b4b6-4d45-aff6-ed570d30fcb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:07:59.317811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20092132-b4b6-4d45-aff6-ed570d30fcb7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:07:59.324506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 222/455] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 6 14:07:59 2022] GET /identity/v3/role_assignments?group.id=e998321c9d7948f891d389130bf1c2f3&scope.project.id=258fdfe108b84b6c82c61b2a2990c11d&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:02.654156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa7c73-0b3c-40fe-8761-e9c39d915cf9 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:02.655212 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa7c73-0b3c-40fe-8761-e9c39d915cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:02.655722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa7c73-0b3c-40fe-8761-e9c39d915cf9 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:02.657622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 234/456] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:02.663593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6c5e4-64b6-4a78-8341-ec4b772211c5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:02.664120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6c5e4-64b6-4a78-8341-ec4b772211c5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:02.664202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6c5e4-64b6-4a78-8341-ec4b772211c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:02.690729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b3d6c5e4-64b6-4a78-8341-ec4b772211c5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:02.692213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b3d6c5e4-64b6-4a78-8341-ec4b772211c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:02.799434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 223/457] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:02.848054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa7c73-0b3c-40fe-8761-e9c39d915cf9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:02.953356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:03.019555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa7c73-0b3c-40fe-8761-e9c39d915cf9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:03.069690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac7b93-9ba1-44e6-a2d8-420603d571f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:03.070541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac7b93-9ba1-44e6-a2d8-420603d571f7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:03.070833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac7b93-9ba1-44e6-a2d8-420603d571f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:03.071110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac7b93-9ba1-44e6-a2d8-420603d571f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:03.091216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac7b93-9ba1-44e6-a2d8-420603d571f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:03.092911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac7b93-9ba1-44e6-a2d8-420603d571f7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-22ac7b93-9ba1-44e6-a2d8-420603d571f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:03.115473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:03.117128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:03.119403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:03.119403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:03.119403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:03.119403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:03.122376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 235/458] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:03.123662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6c5e4-64b6-4a78-8341-ec4b772211c5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:03.235327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:03.263488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6c5e4-64b6-4a78-8341-ec4b772211c5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:03.340507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76116e-705e-4609-92c5-610d1cae8448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:03.341692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76116e-705e-4609-92c5-610d1cae8448 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:03.342231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76116e-705e-4609-92c5-610d1cae8448 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:03.342748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76116e-705e-4609-92c5-610d1cae8448 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:03.344155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76116e-705e-4609-92c5-610d1cae8448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:03.344709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76116e-705e-4609-92c5-610d1cae8448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:03.347293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76116e-705e-4609-92c5-610d1cae8448 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:03.375193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 224/459] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:03.381528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac7b93-9ba1-44e6-a2d8-420603d571f7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:03.518623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52758880-f32a-4919-a03f-ae73b140e771 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:03.519636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52758880-f32a-4919-a03f-ae73b140e771 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:03.520419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52758880-f32a-4919-a03f-ae73b140e771 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:03.523975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 225/460] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:03.536523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4b145-1052-41b4-9e53-41307535de32 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:03.537497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4b145-1052-41b4-9e53-41307535de32 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:03.538309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4b145-1052-41b4-9e53-41307535de32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:03.594096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f3e4b145-1052-41b4-9e53-41307535de32 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:03.601216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f3e4b145-1052-41b4-9e53-41307535de32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:03.613179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac7b93-9ba1-44e6-a2d8-420603d571f7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:03.637564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:03.694598 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c11f-5f30-4d49-a238-2682df4f8d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:03.697567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c11f-5f30-4d49-a238-2682df4f8d68 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:03.699661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c11f-5f30-4d49-a238-2682df4f8d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:03.700227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c11f-5f30-4d49-a238-2682df4f8d68 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:03.701542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92c11f-5f30-4d49-a238-2682df4f8d68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:03.704323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92c11f-5f30-4d49-a238-2682df4f8d68 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:03.772456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 236/461] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:08:03 2022] POST /identity/v3/users => generated 248 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:08:03.812105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 226/462] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:08:03.857245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c11f-5f30-4d49-a238-2682df4f8d68 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:04.115891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c11f-5f30-4d49-a238-2682df4f8d68 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:04.117928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17998-7b86-4ed4-9bff-7c4266ad6694 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:04.118836 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17998-7b86-4ed4-9bff-7c4266ad6694 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:04.119616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17998-7b86-4ed4-9bff-7c4266ad6694 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:04.123758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 227/463] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:04.132823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6ac5e-771e-4599-88ee-21c900e843f6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:04.133721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6ac5e-771e-4599-88ee-21c900e843f6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:04.134966 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6ac5e-771e-4599-88ee-21c900e843f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:04.166142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-87c6ac5e-771e-4599-88ee-21c900e843f6 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:04.168574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87c6ac5e-771e-4599-88ee-21c900e843f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:04.200614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cda97-e2ee-47a1-a75f-8881375c22d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:04.202043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cda97-e2ee-47a1-a75f-8881375c22d5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:04.202521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cda97-e2ee-47a1-a75f-8881375c22d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:04.203510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cda97-e2ee-47a1-a75f-8881375c22d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:04.221211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cda97-e2ee-47a1-a75f-8881375c22d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:04.223828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cda97-e2ee-47a1-a75f-8881375c22d5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7d6cda97-e2ee-47a1-a75f-8881375c22d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:04.251609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:04.253556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:04.255438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:04.255438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:04.255438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:04.255438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:04.255438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 237/464] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:04.256380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e23f11-bf2b-4f8d-abf5-c5f7982a9321 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:04.256629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e23f11-bf2b-4f8d-abf5-c5f7982a9321 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:04.257107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e23f11-bf2b-4f8d-abf5-c5f7982a9321 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:04.260655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 238/465] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:04.263808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e23f11-bf2b-4f8d-abf5-c5f7982a9321 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:04.285567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 228/466] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:04.287457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de23f30-b95a-401c-a986-aac5b719a540 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:04.287909 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de23f30-b95a-401c-a986-aac5b719a540 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:04.288203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de23f30-b95a-401c-a986-aac5b719a540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:04.294788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:04.299396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:04.312349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2de23f30-b95a-401c-a986-aac5b719a540 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:04.313628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2de23f30-b95a-401c-a986-aac5b719a540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:04.429299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 229/467] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:04.431630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de23f30-b95a-401c-a986-aac5b719a540 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:04.465156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e23f11-bf2b-4f8d-abf5-c5f7982a9321 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:04.511903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f0c7e-5c51-4968-9d1c-8fa5c5ae3b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:04.512683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f0c7e-5c51-4968-9d1c-8fa5c5ae3b8a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:04.512976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f0c7e-5c51-4968-9d1c-8fa5c5ae3b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:04.513264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f0c7e-5c51-4968-9d1c-8fa5c5ae3b8a None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:04.514047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f0c7e-5c51-4968-9d1c-8fa5c5ae3b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:04.514359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f0c7e-5c51-4968-9d1c-8fa5c5ae3b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:04.516137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f0c7e-5c51-4968-9d1c-8fa5c5ae3b8a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:04.537760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 239/468] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:04.537760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f0c7e-5c51-4968-9d1c-8fa5c5ae3b8a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:04.646529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:04.666818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de23f30-b95a-401c-a986-aac5b719a540 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:04.679799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f0c7e-5c51-4968-9d1c-8fa5c5ae3b8a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:04.726684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29782ae-a53d-49f4-af6c-866041eff8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:04.727705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29782ae-a53d-49f4-af6c-866041eff8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:04.727705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29782ae-a53d-49f4-af6c-866041eff8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:04.727705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29782ae-a53d-49f4-af6c-866041eff8d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:04.734576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:04.742656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29782ae-a53d-49f4-af6c-866041eff8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:04.745488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29782ae-a53d-49f4-af6c-866041eff8d8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:04.746384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772e934-1902-4520-b3b7-8f1ba89179cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:04.747928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772e934-1902-4520-b3b7-8f1ba89179cc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:04.748848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772e934-1902-4520-b3b7-8f1ba89179cc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:04.749620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772e934-1902-4520-b3b7-8f1ba89179cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-e29782ae-a53d-49f4-af6c-866041eff8d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:04.766878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:04.767944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:04.768890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:04.768890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:04.768890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:04.768890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:04.771129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772e934-1902-4520-b3b7-8f1ba89179cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:04.774413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772e934-1902-4520-b3b7-8f1ba89179cc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:04.776177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 240/469] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:04.777316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29782ae-a53d-49f4-af6c-866041eff8d8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a772e934-1902-4520-b3b7-8f1ba89179cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:04.812135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:04.814752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:04.816489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:04.816489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:04.816489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:04.816489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:04.819678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 230/470] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:04.822197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772e934-1902-4520-b3b7-8f1ba89179cc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:04.904498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29782ae-a53d-49f4-af6c-866041eff8d8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:04.942578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:04.951023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9aef88-ecba-400d-a99b-6913e4882e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:04.952154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aef88-ecba-400d-a99b-6913e4882e19 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:04.952612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aef88-ecba-400d-a99b-6913e4882e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:04.953128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aef88-ecba-400d-a99b-6913e4882e19 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:04.954367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9aef88-ecba-400d-a99b-6913e4882e19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:04.956905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9aef88-ecba-400d-a99b-6913e4882e19 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:04.976898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772e934-1902-4520-b3b7-8f1ba89179cc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:04.991772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 241/471] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:08:04 2022] POST /identity/v3/users => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:08:04.994923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9aef88-ecba-400d-a99b-6913e4882e19 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:05.063627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01112275-10f6-4321-adce-6e6fb0d2b2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:05.064696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01112275-10f6-4321-adce-6e6fb0d2b2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:05.065976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01112275-10f6-4321-adce-6e6fb0d2b2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:05.066471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01112275-10f6-4321-adce-6e6fb0d2b2b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:05.068128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01112275-10f6-4321-adce-6e6fb0d2b2b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:05.068628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01112275-10f6-4321-adce-6e6fb0d2b2b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:05.072388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01112275-10f6-4321-adce-6e6fb0d2b2b2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:05.100062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 231/472] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:05.103450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50c4a5-7cb0-4296-b96b-f88ecd154cd2 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:05.104137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50c4a5-7cb0-4296-b96b-f88ecd154cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:05.104561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50c4a5-7cb0-4296-b96b-f88ecd154cd2 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:05.109872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 232/473] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:05.123283 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f37b9-66c5-406b-89c5-65ddb74073b2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:05.123917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f37b9-66c5-406b-89c5-65ddb74073b2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:05.124398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f37b9-66c5-406b-89c5-65ddb74073b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:05.144320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5c6f37b9-66c5-406b-89c5-65ddb74073b2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:05.146050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5c6f37b9-66c5-406b-89c5-65ddb74073b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:05.152157 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9aef88-ecba-400d-a99b-6913e4882e19 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:05.196714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc91f3f-4c2b-49d7-b132-aa19ce973388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:05.197979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc91f3f-4c2b-49d7-b132-aa19ce973388 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:05.198513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc91f3f-4c2b-49d7-b132-aa19ce973388 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:05.199008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc91f3f-4c2b-49d7-b132-aa19ce973388 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:05.200037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc91f3f-4c2b-49d7-b132-aa19ce973388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:05.200569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc91f3f-4c2b-49d7-b132-aa19ce973388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:05.202422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc91f3f-4c2b-49d7-b132-aa19ce973388 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:05.223660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:05.239145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 242/474] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:05.239145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc91f3f-4c2b-49d7-b132-aa19ce973388 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:05.294849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 233/475] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:05.296183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f37b9-66c5-406b-89c5-65ddb74073b2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:05.336872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:05.441228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc91f3f-4c2b-49d7-b132-aa19ce973388 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:05.493163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f37b9-66c5-406b-89c5-65ddb74073b2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:05.520003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfdca4-dc57-4b2b-8ed1-9caf770e344e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:05.521844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfdca4-dc57-4b2b-8ed1-9caf770e344e None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:05.522399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfdca4-dc57-4b2b-8ed1-9caf770e344e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:05.522920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfdca4-dc57-4b2b-8ed1-9caf770e344e None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:05.523929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfdca4-dc57-4b2b-8ed1-9caf770e344e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:05.526618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfdca4-dc57-4b2b-8ed1-9caf770e344e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:05.568559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9669f4a-d9b4-4075-92a1-23f0294dedd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:05.569913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9669f4a-d9b4-4075-92a1-23f0294dedd7 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:05.570499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9669f4a-d9b4-4075-92a1-23f0294dedd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:05.571126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9669f4a-d9b4-4075-92a1-23f0294dedd7 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:05.572733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 243/476] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:08:05 2022] POST /identity/v3/users => generated 248 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:08:05.573257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9669f4a-d9b4-4075-92a1-23f0294dedd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:05.574651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfdca4-dc57-4b2b-8ed1-9caf770e344e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:05.576512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9669f4a-d9b4-4075-92a1-23f0294dedd7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:05.626875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 234/477] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:08:05 2022] POST /identity/v3/users => generated 249 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:08:05.850372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfdca4-dc57-4b2b-8ed1-9caf770e344e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:05.853885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:05.940880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:05.941766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:05.942279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:05.942763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:05.944130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:05.951192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:05.967993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.sql.core [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 14:08:05.967993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 06 14:08:05.967993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [parameters: {'id': 'e7c2582278da4cc39e6d4bb4804fd91a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 06 14:08:05.967993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80215) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application conn.query(q) Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result.read() Jun 06 14:08:06.000385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return ref.to_dict() Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 14:08:06.001355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application next(self.gen) Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application yield resource Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application next(self.gen) Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise self.value Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application session.commit() Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 14:08:06.002347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.session.flush() Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application compat.raise_( Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 14:08:06.003265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 14:08:06.006300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application conn.query(q) Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result.read() Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 14:08:06.008651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [parameters: {'id': 'e7c2582278da4cc39e6d4bb4804fd91a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:06.018175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 06 14:08:06.020271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:06.022319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 244/478] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 6 14:08:05 2022] POST /identity/v3/roles => generated 144 bytes in 429 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:06.022459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9669f4a-d9b4-4075-92a1-23f0294dedd7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:06.050279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:06.128326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9669f4a-d9b4-4075-92a1-23f0294dedd7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:06.169078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd174-26d2-4a11-8962-3727da7b9620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:06.170052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd174-26d2-4a11-8962-3727da7b9620 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:06.170364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd174-26d2-4a11-8962-3727da7b9620 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:06.170621 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd174-26d2-4a11-8962-3727da7b9620 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:06.183651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd174-26d2-4a11-8962-3727da7b9620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-46bfd174-26d2-4a11-8962-3727da7b9620 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:06.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:06.193603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:06.195056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 235/479] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:08:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:06.199558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:06.463205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073a511-71d8-45c9-9d03-2128a39014d7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:06.554631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4930c8-36de-4ab4-9e67-1f9fc72ad3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:06.555807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4930c8-36de-4ab4-9e67-1f9fc72ad3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:06.557533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4930c8-36de-4ab4-9e67-1f9fc72ad3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:06.558036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4930c8-36de-4ab4-9e67-1f9fc72ad3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:06.559423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4930c8-36de-4ab4-9e67-1f9fc72ad3b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:06.559938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4930c8-36de-4ab4-9e67-1f9fc72ad3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:06.562474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4930c8-36de-4ab4-9e67-1f9fc72ad3b5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:06.578708 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 245/480] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:07.712867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a1fb9-2f0d-4628-a3cb-374a4e0a33fe None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:07.714853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a1fb9-2f0d-4628-a3cb-374a4e0a33fe None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:07.714853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a1fb9-2f0d-4628-a3cb-374a4e0a33fe None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:07.715446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 236/481] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:07.732334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe389f-b874-4805-8fb1-f2841dd093a3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:07.733159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe389f-b874-4805-8fb1-f2841dd093a3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:07.733159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe389f-b874-4805-8fb1-f2841dd093a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:07.760530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-adbe389f-b874-4805-8fb1-f2841dd093a3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:07.761548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-adbe389f-b874-4805-8fb1-f2841dd093a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:07.801552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:07.869055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 246/482] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:07.895292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a1fb9-2f0d-4628-a3cb-374a4e0a33fe None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:08.062467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:08.077226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a1fb9-2f0d-4628-a3cb-374a4e0a33fe None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:08.131314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ee7e4-d8ba-44ae-85f7-0bad9c99fa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:08.132123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ee7e4-d8ba-44ae-85f7-0bad9c99fa30 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:08.132406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ee7e4-d8ba-44ae-85f7-0bad9c99fa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:08.132683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ee7e4-d8ba-44ae-85f7-0bad9c99fa30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:08.153256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ee7e4-d8ba-44ae-85f7-0bad9c99fa30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-277ee7e4-d8ba-44ae-85f7-0bad9c99fa30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:08.161153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:08.162888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:08.164415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 237/483] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:08.170393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe389f-b874-4805-8fb1-f2841dd093a3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:08.315225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3a845-c0ff-445b-a779-ebf71bb78696 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:08.315430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3a845-c0ff-445b-a779-ebf71bb78696 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:08.315719 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3a845-c0ff-445b-a779-ebf71bb78696 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:08.317271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 238/484] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:08:08.333642 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd1035-de5f-4cd9-b3e5-e72e7ba43c7d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:08.334139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd1035-de5f-4cd9-b3e5-e72e7ba43c7d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:08.334623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd1035-de5f-4cd9-b3e5-e72e7ba43c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:08.363939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe389f-b874-4805-8fb1-f2841dd093a3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:08.381688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-95bd1035-de5f-4cd9-b3e5-e72e7ba43c7d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:08.383232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-95bd1035-de5f-4cd9-b3e5-e72e7ba43c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:08.449682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506d7b1-ac9f-4e47-8db9-fb9414c5aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:08.452907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506d7b1-ac9f-4e47-8db9-fb9414c5aa3f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:08.453389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506d7b1-ac9f-4e47-8db9-fb9414c5aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:08.453844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506d7b1-ac9f-4e47-8db9-fb9414c5aa3f None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:08.456562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506d7b1-ac9f-4e47-8db9-fb9414c5aa3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:08.458883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506d7b1-ac9f-4e47-8db9-fb9414c5aa3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:08.460539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506d7b1-ac9f-4e47-8db9-fb9414c5aa3f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:08.475534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 247/485] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:08.482818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506d7b1-ac9f-4e47-8db9-fb9414c5aa3f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:08.545737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 239/486] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:08.545986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:08.574316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd1035-de5f-4cd9-b3e5-e72e7ba43c7d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:08.755896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506d7b1-ac9f-4e47-8db9-fb9414c5aa3f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:08.848620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:08.881595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12c0af-7145-441d-ade4-8b44f4a9f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:08.882953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12c0af-7145-441d-ade4-8b44f4a9f318 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:08.883369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12c0af-7145-441d-ade4-8b44f4a9f318 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:08.883777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12c0af-7145-441d-ade4-8b44f4a9f318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:08.913843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd1035-de5f-4cd9-b3e5-e72e7ba43c7d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:08.924979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12c0af-7145-441d-ade4-8b44f4a9f318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:08.927653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12c0af-7145-441d-ade4-8b44f4a9f318 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2d12c0af-7145-441d-ade4-8b44f4a9f318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:08.985373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:08.987075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:08.987950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:08.987950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:08.987950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:08.987950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:08.987950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 248/487] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 505 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:08.997670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12c0af-7145-441d-ade4-8b44f4a9f318 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:09.019789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10aed91-7cc1-4c4d-b37d-20da2feabf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:09.020990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10aed91-7cc1-4c4d-b37d-20da2feabf2c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.021475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10aed91-7cc1-4c4d-b37d-20da2feabf2c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.022074 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10aed91-7cc1-4c4d-b37d-20da2feabf2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.042594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10aed91-7cc1-4c4d-b37d-20da2feabf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a10aed91-7cc1-4c4d-b37d-20da2feabf2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:09.059558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:09.065633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:09.067190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 240/488] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 488 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:09.075938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10aed91-7cc1-4c4d-b37d-20da2feabf2c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:09.186409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:09.252700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12c0af-7145-441d-ade4-8b44f4a9f318 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:09.282205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10aed91-7cc1-4c4d-b37d-20da2feabf2c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:09.301245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05869d4-b929-4df0-9a92-4a6914396a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:09.301245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05869d4-b929-4df0-9a92-4a6914396a74 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.301245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05869d4-b929-4df0-9a92-4a6914396a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.301245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05869d4-b929-4df0-9a92-4a6914396a74 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.301880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05869d4-b929-4df0-9a92-4a6914396a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:09.302231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05869d4-b929-4df0-9a92-4a6914396a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:09.303948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05869d4-b929-4df0-9a92-4a6914396a74 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:09.319828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 249/489] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:09.322588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f854970-4df1-4cb1-b38f-894ecdbabcff None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.323102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f854970-4df1-4cb1-b38f-894ecdbabcff None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.323608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f854970-4df1-4cb1-b38f-894ecdbabcff None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.333825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 250/490] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:09.335812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f854970-4df1-4cb1-b38f-894ecdbabcff None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:09.354859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d812a-eaeb-4a6f-a844-77c5fa6814a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:09.356507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d812a-eaeb-4a6f-a844-77c5fa6814a1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.357228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d812a-eaeb-4a6f-a844-77c5fa6814a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.358210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d812a-eaeb-4a6f-a844-77c5fa6814a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.360288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d812a-eaeb-4a6f-a844-77c5fa6814a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:09.361120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d812a-eaeb-4a6f-a844-77c5fa6814a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:09.364297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d812a-eaeb-4a6f-a844-77c5fa6814a1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:09.381082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 241/491] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:09.383892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae51ba-5415-43eb-a477-ca69b5247416 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.386284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae51ba-5415-43eb-a477-ca69b5247416 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.387295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae51ba-5415-43eb-a477-ca69b5247416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.396037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:09.482256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f854970-4df1-4cb1-b38f-894ecdbabcff None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:09.521266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:09.527443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922dccc-9733-47c6-9e51-219213c393e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:09.528807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922dccc-9733-47c6-9e51-219213c393e6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.529483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922dccc-9733-47c6-9e51-219213c393e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.529790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922dccc-9733-47c6-9e51-219213c393e6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.538480 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922dccc-9733-47c6-9e51-219213c393e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:09.540013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922dccc-9733-47c6-9e51-219213c393e6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-8922dccc-9733-47c6-9e51-219213c393e6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:09.558905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:09.559930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:09.560890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 251/492] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:09.565898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922dccc-9733-47c6-9e51-219213c393e6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:09.595464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-97ae51ba-5415-43eb-a477-ca69b5247416 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:09.597477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-97ae51ba-5415-43eb-a477-ca69b5247416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:09.685732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 242/493] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:09.685732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a18808-5e80-4a00-b5c2-cf0842ca12f4 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.685732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a18808-5e80-4a00-b5c2-cf0842ca12f4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.685732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a18808-5e80-4a00-b5c2-cf0842ca12f4 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.685732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 243/494] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:09.685732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a18808-5e80-4a00-b5c2-cf0842ca12f4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:09.742036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922dccc-9733-47c6-9e51-219213c393e6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:09.801091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2c239-b02b-4a7c-ba98-995d2990ef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:09.801802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a18808-5e80-4a00-b5c2-cf0842ca12f4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:09.802511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2c239-b02b-4a7c-ba98-995d2990ef64 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.803013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2c239-b02b-4a7c-ba98-995d2990ef64 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.803473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2c239-b02b-4a7c-ba98-995d2990ef64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.821949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2c239-b02b-4a7c-ba98-995d2990ef64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:09.823597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2c239-b02b-4a7c-ba98-995d2990ef64 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-26c2c239-b02b-4a7c-ba98-995d2990ef64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:09.845731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:09.847761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:09.849743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:09.849743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:09.849743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:09.849743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:09.850194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:09.850889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 252/495] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:09.853104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1663d95-e970-4110-b553-4f5de56d8bd5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.853400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1663d95-e970-4110-b553-4f5de56d8bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.853678 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1663d95-e970-4110-b553-4f5de56d8bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.860400 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd9e9e-49e6-4723-a854-1eb21d3a884e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:09.861487 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd9e9e-49e6-4723-a854-1eb21d3a884e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:09.862028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd9e9e-49e6-4723-a854-1eb21d3a884e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:09.862505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd9e9e-49e6-4723-a854-1eb21d3a884e None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:09.864088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd9e9e-49e6-4723-a854-1eb21d3a884e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:09.864631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd9e9e-49e6-4723-a854-1eb21d3a884e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:09.867392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd9e9e-49e6-4723-a854-1eb21d3a884e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:09.872522 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d1663d95-e970-4110-b553-4f5de56d8bd5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:09.873942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d1663d95-e970-4110-b553-4f5de56d8bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:09.919304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 244/496] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 6 14:08:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:09.920172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd9e9e-49e6-4723-a854-1eb21d3a884e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:09.934950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:09.970363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 253/497] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:09.971513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1663d95-e970-4110-b553-4f5de56d8bd5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:10.111908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1663d95-e970-4110-b553-4f5de56d8bd5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:10.127085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd9e9e-49e6-4723-a854-1eb21d3a884e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:10.167371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a1d34-e42f-44e1-a074-49aad9e1b623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:10.168428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a1d34-e42f-44e1-a074-49aad9e1b623 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:10.168758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a1d34-e42f-44e1-a074-49aad9e1b623 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:10.169038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a1d34-e42f-44e1-a074-49aad9e1b623 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:10.169906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a1d34-e42f-44e1-a074-49aad9e1b623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:10.170412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a1d34-e42f-44e1-a074-49aad9e1b623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:10.172831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a1d34-e42f-44e1-a074-49aad9e1b623 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:10.186575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646e49c-f56b-4caa-a49b-641accfb6657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:10.187739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646e49c-f56b-4caa-a49b-641accfb6657 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:10.188134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646e49c-f56b-4caa-a49b-641accfb6657 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:10.188603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646e49c-f56b-4caa-a49b-641accfb6657 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:10.193279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 254/498] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:10.193279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a1d34-e42f-44e1-a074-49aad9e1b623 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:10.209611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646e49c-f56b-4caa-a49b-641accfb6657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9646e49c-f56b-4caa-a49b-641accfb6657 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:10.224913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:10.226840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:10.228974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:10.231504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 245/499] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:10.231504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646e49c-f56b-4caa-a49b-641accfb6657 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:10.255100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:10.323107 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:10.376969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a1d34-e42f-44e1-a074-49aad9e1b623 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:10.405493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646e49c-f56b-4caa-a49b-641accfb6657 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:10.439437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df8355-849b-4310-be02-7efdadfd1338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:10.440470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df8355-849b-4310-be02-7efdadfd1338 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:10.440929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df8355-849b-4310-be02-7efdadfd1338 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:10.441386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df8355-849b-4310-be02-7efdadfd1338 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:10.451688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc55fe-9369-425f-9633-5965e2ab3af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:10.452861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc55fe-9369-425f-9633-5965e2ab3af6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:10.453352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc55fe-9369-425f-9633-5965e2ab3af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:10.453911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc55fe-9369-425f-9633-5965e2ab3af6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:10.471181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc55fe-9369-425f-9633-5965e2ab3af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-18cc55fe-9369-425f-9633-5965e2ab3af6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:10.487545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:10.488696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:10.489525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:10.490602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 246/500] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:10.492347 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc55fe-9369-425f-9633-5965e2ab3af6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:10.555408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df8355-849b-4310-be02-7efdadfd1338 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:10.558919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df8355-849b-4310-be02-7efdadfd1338 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:10.623224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:10.675994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc55fe-9369-425f-9633-5965e2ab3af6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:10.698732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:10.706335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 255/501] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:10.714028 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df8355-849b-4310-be02-7efdadfd1338 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:10.740921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbcfba-014d-40b6-889e-968d84a38596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:10.741593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbcfba-014d-40b6-889e-968d84a38596 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:10.741906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbcfba-014d-40b6-889e-968d84a38596 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:10.742187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbcfba-014d-40b6-889e-968d84a38596 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:10.753599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbcfba-014d-40b6-889e-968d84a38596 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:10.755124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbcfba-014d-40b6-889e-968d84a38596 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d1cbcfba-014d-40b6-889e-968d84a38596 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:10.780843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:10.782492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 247/502] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 6 14:08:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:10.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbcfba-014d-40b6-889e-968d84a38596 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:10.935544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbcfba-014d-40b6-889e-968d84a38596 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:10.939928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df8355-849b-4310-be02-7efdadfd1338 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:10.992682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040346e-6c3a-40b2-9665-d6d8c87cd7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:10.994151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6d887-68ac-4f85-a805-1396b4bdd9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:10.994598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040346e-6c3a-40b2-9665-d6d8c87cd7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:10.995088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040346e-6c3a-40b2-9665-d6d8c87cd7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:10.995245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6d887-68ac-4f85-a805-1396b4bdd9bd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:10.995563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6d887-68ac-4f85-a805-1396b4bdd9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:10.995716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040346e-6c3a-40b2-9665-d6d8c87cd7f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:10.996032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6d887-68ac-4f85-a805-1396b4bdd9bd None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:10.996941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6d887-68ac-4f85-a805-1396b4bdd9bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:10.997196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040346e-6c3a-40b2-9665-d6d8c87cd7f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:10.997455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6d887-68ac-4f85-a805-1396b4bdd9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:10.997693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040346e-6c3a-40b2-9665-d6d8c87cd7f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:10.999222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6d887-68ac-4f85-a805-1396b4bdd9bd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:11.000233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040346e-6c3a-40b2-9665-d6d8c87cd7f3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:11.001394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:11.017913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 256/503] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:11.018913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6d887-68ac-4f85-a805-1396b4bdd9bd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:11.021582 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 248/504] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:11.023084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040346e-6c3a-40b2-9665-d6d8c87cd7f3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:11.082740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:11.179963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6d887-68ac-4f85-a805-1396b4bdd9bd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:11.180889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040346e-6c3a-40b2-9665-d6d8c87cd7f3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:11.224909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59da82-675a-4ecf-a931-d8fa2a5842f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:11.225208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f7ac-e6cb-4d21-952f-69cb97183356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:11.225506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f7ac-e6cb-4d21-952f-69cb97183356 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:11.225825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f7ac-e6cb-4d21-952f-69cb97183356 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:11.226052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59da82-675a-4ecf-a931-d8fa2a5842f5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:11.226212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f7ac-e6cb-4d21-952f-69cb97183356 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:11.226524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59da82-675a-4ecf-a931-d8fa2a5842f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:11.226879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59da82-675a-4ecf-a931-d8fa2a5842f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:11.227034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f7ac-e6cb-4d21-952f-69cb97183356 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:11.227273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f7ac-e6cb-4d21-952f-69cb97183356 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:11.228839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697f7ac-e6cb-4d21-952f-69cb97183356 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:11.241194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59da82-675a-4ecf-a931-d8fa2a5842f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:11.242880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59da82-675a-4ecf-a931-d8fa2a5842f5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:11.249609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 249/505] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 6 14:08:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:11.252918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f7ac-e6cb-4d21-952f-69cb97183356 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0b59da82-675a-4ecf-a931-d8fa2a5842f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:11.264641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:11.266707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:11.268769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:11.268769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:11.268769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:11.268769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:11.268769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 257/506] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:11.268769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59da82-675a-4ecf-a931-d8fa2a5842f5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:11.335235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:11.382991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:11.395448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f7ac-e6cb-4d21-952f-69cb97183356 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:11.403729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59da82-675a-4ecf-a931-d8fa2a5842f5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:11.433797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84960db-099e-4af9-8068-4514e58b03db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:11.434650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84960db-099e-4af9-8068-4514e58b03db None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:11.434941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84960db-099e-4af9-8068-4514e58b03db None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:11.435306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84960db-099e-4af9-8068-4514e58b03db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:11.447231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84960db-099e-4af9-8068-4514e58b03db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:11.448937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84960db-099e-4af9-8068-4514e58b03db None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d84960db-099e-4af9-8068-4514e58b03db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:11.469969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:11.471713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:11.473232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:11.473232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:11.473232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:11.473232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:11.473232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 250/507] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:11.474499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36a533-af86-4c74-8ee2-46d445262f28 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:11.474499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36a533-af86-4c74-8ee2-46d445262f28 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:11.475491 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36a533-af86-4c74-8ee2-46d445262f28 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:11.477850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 251/508] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:11.479524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36a533-af86-4c74-8ee2-46d445262f28 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:11.483223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d036f8-1560-4184-bdbd-0f8eaea81445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:11.484626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d036f8-1560-4184-bdbd-0f8eaea81445 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:11.484982 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d036f8-1560-4184-bdbd-0f8eaea81445 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:11.485334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d036f8-1560-4184-bdbd-0f8eaea81445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:11.499993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d036f8-1560-4184-bdbd-0f8eaea81445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:11.501990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d036f8-1560-4184-bdbd-0f8eaea81445 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-02d036f8-1560-4184-bdbd-0f8eaea81445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:11.535733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:11.537405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:11.539417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:11.539417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:11.539417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:11.539417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:11.539417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 258/509] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:11.539924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d036f8-1560-4184-bdbd-0f8eaea81445 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:11.643049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36a533-af86-4c74-8ee2-46d445262f28 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:11.675810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:11.682537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d036f8-1560-4184-bdbd-0f8eaea81445 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:11.691465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc8aa0-eb1a-4188-8e8e-e4f4f32b089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:11.692249 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc8aa0-eb1a-4188-8e8e-e4f4f32b089d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:11.692648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc8aa0-eb1a-4188-8e8e-e4f4f32b089d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:11.693014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc8aa0-eb1a-4188-8e8e-e4f4f32b089d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:11.707099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:11.743332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28503ab-91b4-4a86-a601-823c778e5600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:11.744338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28503ab-91b4-4a86-a601-823c778e5600 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:11.744753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28503ab-91b4-4a86-a601-823c778e5600 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:11.745148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28503ab-91b4-4a86-a601-823c778e5600 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:11.746065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28503ab-91b4-4a86-a601-823c778e5600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:11.746474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28503ab-91b4-4a86-a601-823c778e5600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:11.748777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28503ab-91b4-4a86-a601-823c778e5600 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:11.767713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 259/510] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:11.768986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28503ab-91b4-4a86-a601-823c778e5600 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:11.782518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc8aa0-eb1a-4188-8e8e-e4f4f32b089d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:11.784127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc8aa0-eb1a-4188-8e8e-e4f4f32b089d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:11.913479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28503ab-91b4-4a86-a601-823c778e5600 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:11.943820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 252/511] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:11.943820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4480df7-f2c8-4d2b-b1ae-5ddd96fe01aa None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:11.943820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4480df7-f2c8-4d2b-b1ae-5ddd96fe01aa None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:11.943820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4480df7-f2c8-4d2b-b1ae-5ddd96fe01aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:11.972601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e4480df7-f2c8-4d2b-b1ae-5ddd96fe01aa None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:11.973843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e4480df7-f2c8-4d2b-b1ae-5ddd96fe01aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:11.978107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31644661-2991-448e-806e-4bcbb376dc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:11.979253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31644661-2991-448e-806e-4bcbb376dc94 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:11.979757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31644661-2991-448e-806e-4bcbb376dc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:11.980256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31644661-2991-448e-806e-4bcbb376dc94 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:11.981589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31644661-2991-448e-806e-4bcbb376dc94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:11.982118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31644661-2991-448e-806e-4bcbb376dc94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:11.984931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31644661-2991-448e-806e-4bcbb376dc94 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:12.006849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 260/512] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:12.008271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31644661-2991-448e-806e-4bcbb376dc94 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:12.018080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:12.041415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:12.063763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 253/513] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:12.064819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4480df7-f2c8-4d2b-b1ae-5ddd96fe01aa None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:12.158630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31644661-2991-448e-806e-4bcbb376dc94 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:12.200713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4480df7-f2c8-4d2b-b1ae-5ddd96fe01aa None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:12.205300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00512aaf-0812-41f8-89ac-3662b4c6d783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:12.206310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00512aaf-0812-41f8-89ac-3662b4c6d783 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:12.206776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00512aaf-0812-41f8-89ac-3662b4c6d783 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:12.207225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00512aaf-0812-41f8-89ac-3662b4c6d783 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:12.208478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00512aaf-0812-41f8-89ac-3662b4c6d783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:12.208982 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00512aaf-0812-41f8-89ac-3662b4c6d783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:12.211484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00512aaf-0812-41f8-89ac-3662b4c6d783 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:12.238565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 261/514] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:12.238565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00512aaf-0812-41f8-89ac-3662b4c6d783 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:12.259721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a939016-be48-46d7-a1fc-ab59530ae5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:12.260847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a939016-be48-46d7-a1fc-ab59530ae5ab None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:12.261247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a939016-be48-46d7-a1fc-ab59530ae5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:12.261672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a939016-be48-46d7-a1fc-ab59530ae5ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:12.273608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a939016-be48-46d7-a1fc-ab59530ae5ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:12.276764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a939016-be48-46d7-a1fc-ab59530ae5ab None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2a939016-be48-46d7-a1fc-ab59530ae5ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:12.302377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:12.304181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 254/515] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:12.305894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a939016-be48-46d7-a1fc-ab59530ae5ab None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:12.349321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:12.364140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:12.392616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00512aaf-0812-41f8-89ac-3662b4c6d783 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:12.441139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fa0ad-7907-410e-ac04-746543211833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:12.442087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fa0ad-7907-410e-ac04-746543211833 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:12.442456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fa0ad-7907-410e-ac04-746543211833 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:12.442855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fa0ad-7907-410e-ac04-746543211833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:12.445805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a939016-be48-46d7-a1fc-ab59530ae5ab None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:12.456543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fa0ad-7907-410e-ac04-746543211833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:12.458540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fa0ad-7907-410e-ac04-746543211833 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2a1fa0ad-7907-410e-ac04-746543211833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:12.481053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:12.482747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:12.484672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:12.484672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:12.484672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:12.484672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:12.484672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 262/516] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:12.485187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fa0ad-7907-410e-ac04-746543211833 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:12.493919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78533040-d184-425f-82d9-4f95c1cf79f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:12.494827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78533040-d184-425f-82d9-4f95c1cf79f9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:12.495173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78533040-d184-425f-82d9-4f95c1cf79f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:12.495513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78533040-d184-425f-82d9-4f95c1cf79f9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:12.507335 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78533040-d184-425f-82d9-4f95c1cf79f9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:12.510758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78533040-d184-425f-82d9-4f95c1cf79f9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-78533040-d184-425f-82d9-4f95c1cf79f9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:12.535270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:12.537129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 255/517] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:12.538959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78533040-d184-425f-82d9-4f95c1cf79f9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:12.632464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fa0ad-7907-410e-ac04-746543211833 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:12.676524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:12.683410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:12.690848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78533040-d184-425f-82d9-4f95c1cf79f9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:12.691154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819f46f-e7eb-4517-9a9c-6839718e29c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:12.691846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819f46f-e7eb-4517-9a9c-6839718e29c6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:12.692172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819f46f-e7eb-4517-9a9c-6839718e29c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:12.692491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819f46f-e7eb-4517-9a9c-6839718e29c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:12.705040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819f46f-e7eb-4517-9a9c-6839718e29c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:12.706938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819f46f-e7eb-4517-9a9c-6839718e29c6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-6819f46f-e7eb-4517-9a9c-6839718e29c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:12.730278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:12.732051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:12.734449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:12.734449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:12.734449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:12.734449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:12.734449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 263/518] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:12.735078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ff74c-2d6c-4048-bbe9-941269a35011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:12.735206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819f46f-e7eb-4517-9a9c-6839718e29c6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:12.735315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ff74c-2d6c-4048-bbe9-941269a35011 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:12.735712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ff74c-2d6c-4048-bbe9-941269a35011 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:12.736202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ff74c-2d6c-4048-bbe9-941269a35011 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:12.749829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ff74c-2d6c-4048-bbe9-941269a35011 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:12.751811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ff74c-2d6c-4048-bbe9-941269a35011 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9a4ff74c-2d6c-4048-bbe9-941269a35011 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:12.772072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:12.773691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 256/519] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:12.777910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ff74c-2d6c-4048-bbe9-941269a35011 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:12.870811 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819f46f-e7eb-4517-9a9c-6839718e29c6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:12.912542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bdb56-7744-4bfe-8dad-d5a141a7097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:12.913200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bdb56-7744-4bfe-8dad-d5a141a7097c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:12.913414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ff74c-2d6c-4048-bbe9-941269a35011 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:12.913568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bdb56-7744-4bfe-8dad-d5a141a7097c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:12.913770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bdb56-7744-4bfe-8dad-d5a141a7097c None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:12.914576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bdb56-7744-4bfe-8dad-d5a141a7097c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:12.914878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bdb56-7744-4bfe-8dad-d5a141a7097c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:12.916890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bdb56-7744-4bfe-8dad-d5a141a7097c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:12.937495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 264/520] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:12.938406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bdb56-7744-4bfe-8dad-d5a141a7097c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:12.955381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693522e-0fd0-469c-969a-652f8e49ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:12.956179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693522e-0fd0-469c-969a-652f8e49ec77 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:12.956350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693522e-0fd0-469c-969a-652f8e49ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:12.956747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693522e-0fd0-469c-969a-652f8e49ec77 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:12.957326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693522e-0fd0-469c-969a-652f8e49ec77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:12.957641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693522e-0fd0-469c-969a-652f8e49ec77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:12.959086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693522e-0fd0-469c-969a-652f8e49ec77 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:12.973816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 257/521] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:12.974539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693522e-0fd0-469c-969a-652f8e49ec77 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:13.001439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:13.018218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:13.096154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bdb56-7744-4bfe-8dad-d5a141a7097c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:13.131041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693522e-0fd0-469c-969a-652f8e49ec77 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:13.135474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae0786-45b0-4bbb-86f3-b655ddfbf75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:13.136164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae0786-45b0-4bbb-86f3-b655ddfbf75f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:13.136481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae0786-45b0-4bbb-86f3-b655ddfbf75f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:13.136750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae0786-45b0-4bbb-86f3-b655ddfbf75f None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:13.137601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae0786-45b0-4bbb-86f3-b655ddfbf75f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:13.137939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae0786-45b0-4bbb-86f3-b655ddfbf75f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:13.139416 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae0786-45b0-4bbb-86f3-b655ddfbf75f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:13.170777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755a9f1-956f-4c52-a536-f3d321dc9c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:13.171444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755a9f1-956f-4c52-a536-f3d321dc9c37 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:13.171729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755a9f1-956f-4c52-a536-f3d321dc9c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:13.172008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755a9f1-956f-4c52-a536-f3d321dc9c37 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:13.172743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755a9f1-956f-4c52-a536-f3d321dc9c37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:13.173049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755a9f1-956f-4c52-a536-f3d321dc9c37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:13.174468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755a9f1-956f-4c52-a536-f3d321dc9c37 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:13.179120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 265/522] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:13.179725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae0786-45b0-4bbb-86f3-b655ddfbf75f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:13.188183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 258/523] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:13.188970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755a9f1-956f-4c52-a536-f3d321dc9c37 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:13.281144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:13.297439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae0786-45b0-4bbb-86f3-b655ddfbf75f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:13.299663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:13.329595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755a9f1-956f-4c52-a536-f3d321dc9c37 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:13.342677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0854f7-20d3-41c8-a6b9-2fc6d64df098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:13.343379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0854f7-20d3-41c8-a6b9-2fc6d64df098 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:13.343669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0854f7-20d3-41c8-a6b9-2fc6d64df098 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:13.343942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0854f7-20d3-41c8-a6b9-2fc6d64df098 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:13.344837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0854f7-20d3-41c8-a6b9-2fc6d64df098 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:13.345149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0854f7-20d3-41c8-a6b9-2fc6d64df098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:13.346833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0854f7-20d3-41c8-a6b9-2fc6d64df098 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:13.368948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 266/524] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 6 14:08:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:13.369955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0854f7-20d3-41c8-a6b9-2fc6d64df098 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:13.381806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fdc69-a028-4e8a-8a10-81c4b9b2d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:13.382551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fdc69-a028-4e8a-8a10-81c4b9b2d0bc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:13.383053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fdc69-a028-4e8a-8a10-81c4b9b2d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:13.383549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fdc69-a028-4e8a-8a10-81c4b9b2d0bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:13.386777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fdc69-a028-4e8a-8a10-81c4b9b2d0bc None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=f41a357957bc47739b7bd6f21d159d74) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:13.387336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fdc69-a028-4e8a-8a10-81c4b9b2d0bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:13.390186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fdc69-a028-4e8a-8a10-81c4b9b2d0bc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:13.401465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 259/525] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:13 2022] GET /identity/v3/role_assignments?user.id=f41a357957bc47739b7bd6f21d159d74&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:13.402916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fdc69-a028-4e8a-8a10-81c4b9b2d0bc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:13.520048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0854f7-20d3-41c8-a6b9-2fc6d64df098 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:13.580536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fdc69-a028-4e8a-8a10-81c4b9b2d0bc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:13.580814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5f630-7d1f-4158-b444-07e66bb6e257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:13.581495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5f630-7d1f-4158-b444-07e66bb6e257 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:13.581950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5f630-7d1f-4158-b444-07e66bb6e257 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:13.582101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5f630-7d1f-4158-b444-07e66bb6e257 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:13.591137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5f630-7d1f-4158-b444-07e66bb6e257 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:13.592718 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5f630-7d1f-4158-b444-07e66bb6e257 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-abe5f630-7d1f-4158-b444-07e66bb6e257 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:13.609820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:13.611064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 267/526] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:13.612488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5f630-7d1f-4158-b444-07e66bb6e257 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:13.642560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:13.663419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ca772-5222-41a8-a520-754253304d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:13.664095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ca772-5222-41a8-a520-754253304d8f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:13.664385 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ca772-5222-41a8-a520-754253304d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:13.664671 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ca772-5222-41a8-a520-754253304d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:13.670265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:13.764488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5f630-7d1f-4158-b444-07e66bb6e257 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:13.769141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ca772-5222-41a8-a520-754253304d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:13.771198 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ca772-5222-41a8-a520-754253304d8f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:13.815249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34183dc4-bc95-4cc8-ad33-dcc699c31260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:13.816310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34183dc4-bc95-4cc8-ad33-dcc699c31260 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:13.816713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34183dc4-bc95-4cc8-ad33-dcc699c31260 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:13.817173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34183dc4-bc95-4cc8-ad33-dcc699c31260 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:13.818505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34183dc4-bc95-4cc8-ad33-dcc699c31260 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:13.821112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34183dc4-bc95-4cc8-ad33-dcc699c31260 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:13.865851 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 268/527] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:08:13 2022] POST /identity/v3/users => generated 247 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:08:13.865851 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34183dc4-bc95-4cc8-ad33-dcc699c31260 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:14.011921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 260/528] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:14.013698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ca772-5222-41a8-a520-754253304d8f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:14.020196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:14.067137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:14.091743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34183dc4-bc95-4cc8-ad33-dcc699c31260 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:14.146124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a9e6d-dcca-4bf2-8e41-8e86bdeefcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:14.146754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a9e6d-dcca-4bf2-8e41-8e86bdeefcaf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:14.147047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a9e6d-dcca-4bf2-8e41-8e86bdeefcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:14.147330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a9e6d-dcca-4bf2-8e41-8e86bdeefcaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:14.176118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ca772-5222-41a8-a520-754253304d8f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:14.217698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b5aed-430c-4f74-89bb-c603065ea6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:14.218773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b5aed-430c-4f74-89bb-c603065ea6aa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:14.219245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b5aed-430c-4f74-89bb-c603065ea6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:14.219730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b5aed-430c-4f74-89bb-c603065ea6aa None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:14.220750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b5aed-430c-4f74-89bb-c603065ea6aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:14.221266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b5aed-430c-4f74-89bb-c603065ea6aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:14.222992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b5aed-430c-4f74-89bb-c603065ea6aa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:14.235701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a9e6d-dcca-4bf2-8e41-8e86bdeefcaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:14.237359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a9e6d-dcca-4bf2-8e41-8e86bdeefcaf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:14.244613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 261/529] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:14.245503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b5aed-430c-4f74-89bb-c603065ea6aa None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:14.363713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:14.413059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 269/530] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:14.414066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a9e6d-dcca-4bf2-8e41-8e86bdeefcaf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:14.433787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:14.443003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b5aed-430c-4f74-89bb-c603065ea6aa None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:14.522401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2f6a5-16f2-4403-a3a9-966d9b7be6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:14.524087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2f6a5-16f2-4403-a3a9-966d9b7be6ff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:14.525168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2f6a5-16f2-4403-a3a9-966d9b7be6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:14.525644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2f6a5-16f2-4403-a3a9-966d9b7be6ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:14.550420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2f6a5-16f2-4403-a3a9-966d9b7be6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:14.554239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2f6a5-16f2-4403-a3a9-966d9b7be6ff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ffe2f6a5-16f2-4403-a3a9-966d9b7be6ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:14.599444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:14.601031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:14.603494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:14.603494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:14.603494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:14.603494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:14.603494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 262/531] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:14.604706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2f6a5-16f2-4403-a3a9-966d9b7be6ff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:14.644337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a9e6d-dcca-4bf2-8e41-8e86bdeefcaf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:14.715155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e088d1-1e9f-402c-b9a8-1ca191662661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:14.716746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e088d1-1e9f-402c-b9a8-1ca191662661 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:14.717222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e088d1-1e9f-402c-b9a8-1ca191662661 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:14.717688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e088d1-1e9f-402c-b9a8-1ca191662661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:14.719735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e088d1-1e9f-402c-b9a8-1ca191662661 None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=0b82a920df4042dd97314350aed4d7a5) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:14.720663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e088d1-1e9f-402c-b9a8-1ca191662661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:14.723639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e088d1-1e9f-402c-b9a8-1ca191662661 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:14.737874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 270/532] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:14 2022] GET /identity/v3/role_assignments?user.id=0b82a920df4042dd97314350aed4d7a5&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:14.739666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e088d1-1e9f-402c-b9a8-1ca191662661 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:14.761904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2f6a5-16f2-4403-a3a9-966d9b7be6ff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:14.831906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:14.865094 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49213e-999e-4812-9070-871b35c6d5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:14.866395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49213e-999e-4812-9070-871b35c6d5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:14.866873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49213e-999e-4812-9070-871b35c6d5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:14.867180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49213e-999e-4812-9070-871b35c6d5d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:14.936311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49213e-999e-4812-9070-871b35c6d5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:14.939105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49213e-999e-4812-9070-871b35c6d5d7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:14.947791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e088d1-1e9f-402c-b9a8-1ca191662661 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:14.975496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ed49213e-999e-4812-9070-871b35c6d5d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:14.986349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:14.987951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:14.989572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:14.989572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:14.989572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:14.989572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:14.989572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 263/533] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:14.994063 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49213e-999e-4812-9070-871b35c6d5d7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:15.029448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd4dda-17a9-43d6-b14c-f765e17ce3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:15.031817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd4dda-17a9-43d6-b14c-f765e17ce3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:15.032580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd4dda-17a9-43d6-b14c-f765e17ce3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:15.033277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd4dda-17a9-43d6-b14c-f765e17ce3a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:15.035126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd4dda-17a9-43d6-b14c-f765e17ce3a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:15.035965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd4dda-17a9-43d6-b14c-f765e17ce3a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:15.038749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd4dda-17a9-43d6-b14c-f765e17ce3a3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:15.064251 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 271/534] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:15.078774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd4dda-17a9-43d6-b14c-f765e17ce3a3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:15.173626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49213e-999e-4812-9070-871b35c6d5d7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:15.262040 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcef0f7-8dbc-48f7-9da6-2ed3e8eda281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:15.262861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcef0f7-8dbc-48f7-9da6-2ed3e8eda281 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:15.263353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcef0f7-8dbc-48f7-9da6-2ed3e8eda281 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:15.263836 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcef0f7-8dbc-48f7-9da6-2ed3e8eda281 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:15.265415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcef0f7-8dbc-48f7-9da6-2ed3e8eda281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:15.265993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcef0f7-8dbc-48f7-9da6-2ed3e8eda281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:15.268753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcef0f7-8dbc-48f7-9da6-2ed3e8eda281 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:15.292177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 264/535] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:15.299377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcef0f7-8dbc-48f7-9da6-2ed3e8eda281 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:15.427744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:15.435646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:15.445338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd4dda-17a9-43d6-b14c-f765e17ce3a3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:15.543909 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcef0f7-8dbc-48f7-9da6-2ed3e8eda281 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:15.579891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f023db0-5a1d-417c-885e-6b9558291b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:15.581283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f023db0-5a1d-417c-885e-6b9558291b74 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:15.581809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f023db0-5a1d-417c-885e-6b9558291b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:15.582312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f023db0-5a1d-417c-885e-6b9558291b74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:15.606667 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f023db0-5a1d-417c-885e-6b9558291b74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:15.608538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f023db0-5a1d-417c-885e-6b9558291b74 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7f023db0-5a1d-417c-885e-6b9558291b74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:15.637736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:15.638668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:15.639523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:15.639523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:15.639523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:15.639523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:15.639523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:15.639523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:15.639523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:15.639523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:15.642166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 272/536] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 564 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:15.644568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e5790-1862-4cdd-a2e7-0209df8cc31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:15.646087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e5790-1862-4cdd-a2e7-0209df8cc31d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:15.646600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e5790-1862-4cdd-a2e7-0209df8cc31d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:15.647099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e5790-1862-4cdd-a2e7-0209df8cc31d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:15.648990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f023db0-5a1d-417c-885e-6b9558291b74 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:15.664765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e5790-1862-4cdd-a2e7-0209df8cc31d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:15.672145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e5790-1862-4cdd-a2e7-0209df8cc31d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-186e5790-1862-4cdd-a2e7-0209df8cc31d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:15.730848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:15.731782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:15.732626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:15.732626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:15.732626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:15.732626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:15.732626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:15.732626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:15.732626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:15.732626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:15.738635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 265/537] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:15.740558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e5790-1862-4cdd-a2e7-0209df8cc31d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:15.957144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f023db0-5a1d-417c-885e-6b9558291b74 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:15.992791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e5790-1862-4cdd-a2e7-0209df8cc31d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:16.041685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:16.067868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac2d51-ceb1-435c-8a18-7f81a86b197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:16.069014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac2d51-ceb1-435c-8a18-7f81a86b197b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:16.069525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac2d51-ceb1-435c-8a18-7f81a86b197b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:16.070072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac2d51-ceb1-435c-8a18-7f81a86b197b None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:16.072375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac2d51-ceb1-435c-8a18-7f81a86b197b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:16.072940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac2d51-ceb1-435c-8a18-7f81a86b197b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:16.076154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac2d51-ceb1-435c-8a18-7f81a86b197b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:16.100643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 266/538] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:16.106315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fef430-98ab-4642-a0f5-b350f8a1daab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:16.107559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fef430-98ab-4642-a0f5-b350f8a1daab None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:16.108076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fef430-98ab-4642-a0f5-b350f8a1daab None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:16.108568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fef430-98ab-4642-a0f5-b350f8a1daab None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:16.110477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fef430-98ab-4642-a0f5-b350f8a1daab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:16.111044 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fef430-98ab-4642-a0f5-b350f8a1daab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:16.114307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac2d51-ceb1-435c-8a18-7f81a86b197b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:16.115740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fef430-98ab-4642-a0f5-b350f8a1daab None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:16.143061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 273/539] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:16.152333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fef430-98ab-4642-a0f5-b350f8a1daab None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:16.222192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:16.426269 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac2d51-ceb1-435c-8a18-7f81a86b197b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:16.433341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fef430-98ab-4642-a0f5-b350f8a1daab None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:16.523824 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494c323-cab6-4cc4-a5c6-b5fbce26b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:16.525552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494c323-cab6-4cc4-a5c6-b5fbce26b45d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:16.526003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494c323-cab6-4cc4-a5c6-b5fbce26b45d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:16.526405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494c323-cab6-4cc4-a5c6-b5fbce26b45d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:16.527778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494c323-cab6-4cc4-a5c6-b5fbce26b45d None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=720c995318a24e7b859fa82febd8178b) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:16.528314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494c323-cab6-4cc4-a5c6-b5fbce26b45d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:16.531074 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494c323-cab6-4cc4-a5c6-b5fbce26b45d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:16.532034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba219d00-b1a1-4866-ae76-6bd4f9c1021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:16.532951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba219d00-b1a1-4866-ae76-6bd4f9c1021f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:16.533443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba219d00-b1a1-4866-ae76-6bd4f9c1021f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:16.533985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba219d00-b1a1-4866-ae76-6bd4f9c1021f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:16.535056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba219d00-b1a1-4866-ae76-6bd4f9c1021f None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=59cd9d922204496d8ea185716b832481) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:16.535604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba219d00-b1a1-4866-ae76-6bd4f9c1021f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:16.537328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba219d00-b1a1-4866-ae76-6bd4f9c1021f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:16.545463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 267/540] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:16 2022] GET /identity/v3/role_assignments?user.id=720c995318a24e7b859fa82febd8178b&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 259 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:16.568101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 274/541] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:16 2022] GET /identity/v3/role_assignments?user.id=59cd9d922204496d8ea185716b832481&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 259 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:17.329602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b585-8a0f-4031-a6e6-72e6853f9b52 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:17.330411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b585-8a0f-4031-a6e6-72e6853f9b52 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:17.331191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b585-8a0f-4031-a6e6-72e6853f9b52 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:17.334468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 268/542] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:17.340560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e885842-af24-4984-b3fd-31f5f9c91cfb None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:17.341402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e885842-af24-4984-b3fd-31f5f9c91cfb None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:17.341909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e885842-af24-4984-b3fd-31f5f9c91cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:17.372802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1e885842-af24-4984-b3fd-31f5f9c91cfb None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:17.374409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1e885842-af24-4984-b3fd-31f5f9c91cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:17.483193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 275/543] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:17.521168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b585-8a0f-4031-a6e6-72e6853f9b52 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:17.653095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:17.791767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b585-8a0f-4031-a6e6-72e6853f9b52 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:17.865624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3963e-2f8d-458f-9e0c-471775975033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:17.867274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3963e-2f8d-458f-9e0c-471775975033 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:17.867967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3963e-2f8d-458f-9e0c-471775975033 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:17.868711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3963e-2f8d-458f-9e0c-471775975033 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:17.875384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ab7f-aa4c-4779-89bc-0ff8fe5f49bd None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:17.876087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ab7f-aa4c-4779-89bc-0ff8fe5f49bd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:17.876591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ab7f-aa4c-4779-89bc-0ff8fe5f49bd None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:17.878568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 276/544] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:17.896172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3963e-2f8d-458f-9e0c-471775975033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:17.917996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039eb5d-f466-4731-b002-9ef2adc5ca0c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1df3963e-2f8d-458f-9e0c-471775975033 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:17.918169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:17.919038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:17.919835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:17.920464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039eb5d-f466-4731-b002-9ef2adc5ca0c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:17.920464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039eb5d-f466-4731-b002-9ef2adc5ca0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:17.921115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 269/545] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:17.931739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3963e-2f8d-458f-9e0c-471775975033 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:17.979761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0039eb5d-f466-4731-b002-9ef2adc5ca0c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:17.982004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0039eb5d-f466-4731-b002-9ef2adc5ca0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:18.022666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:18.152311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3963e-2f8d-458f-9e0c-471775975033 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:18.157122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:18.227205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987d785-02a0-4914-8549-4ed9c62ddfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:18.228734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987d785-02a0-4914-8549-4ed9c62ddfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:18.229239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987d785-02a0-4914-8549-4ed9c62ddfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:18.229721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987d785-02a0-4914-8549-4ed9c62ddfe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:18.231418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987d785-02a0-4914-8549-4ed9c62ddfe0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:18.231960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987d785-02a0-4914-8549-4ed9c62ddfe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:18.235130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987d785-02a0-4914-8549-4ed9c62ddfe0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:18.253057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 270/546] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:18.263443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987d785-02a0-4914-8549-4ed9c62ddfe0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:18.306517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 277/547] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:18.352045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039eb5d-f466-4731-b002-9ef2adc5ca0c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:18.563993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987d785-02a0-4914-8549-4ed9c62ddfe0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:18.600502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039eb5d-f466-4731-b002-9ef2adc5ca0c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:18.613182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35e4cb-90d4-4b18-bbdf-73b35ece0efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:18.614486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35e4cb-90d4-4b18-bbdf-73b35ece0efb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:18.615011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35e4cb-90d4-4b18-bbdf-73b35ece0efb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:18.616151 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35e4cb-90d4-4b18-bbdf-73b35ece0efb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:18.630981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35e4cb-90d4-4b18-bbdf-73b35ece0efb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:18.633021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35e4cb-90d4-4b18-bbdf-73b35ece0efb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:18.653824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9f395-00ad-4774-b2fb-96cac205b089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:18.654892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9f395-00ad-4774-b2fb-96cac205b089 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:18.655348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9f395-00ad-4774-b2fb-96cac205b089 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:18.655762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9f395-00ad-4774-b2fb-96cac205b089 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-fb35e4cb-90d4-4b18-bbdf-73b35ece0efb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:18.657742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:18.658636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:18.659497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:18.659497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:18.659497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:18.659497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:18.661855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 271/548] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:18.663658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fa72-82e5-493c-a71a-93e4eefed941 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:18.665743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fa72-82e5-493c-a71a-93e4eefed941 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:18.668593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fa72-82e5-493c-a71a-93e4eefed941 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:18.670419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 272/549] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:18.671178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280fa72-82e5-493c-a71a-93e4eefed941 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:18.684226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9f395-00ad-4774-b2fb-96cac205b089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a8e9f395-00ad-4774-b2fb-96cac205b089 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:18.693429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:18.694925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:18.695739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:18.698636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 278/550] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:18.700303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8f761-fc44-42f5-abab-044fd7d2f40e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:18.700787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8f761-fc44-42f5-abab-044fd7d2f40e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:18.701257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8f761-fc44-42f5-abab-044fd7d2f40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:18.708524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:18.715549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:18.727571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-18b8f761-fc44-42f5-abab-044fd7d2f40e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:18.730540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-18b8f761-fc44-42f5-abab-044fd7d2f40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:18.826313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280fa72-82e5-493c-a71a-93e4eefed941 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:18.866018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 279/551] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:18.868091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8f761-fc44-42f5-abab-044fd7d2f40e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:18.871711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0a0ea-4ec8-4380-8f3b-fb4b64ab9ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:18.872714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0a0ea-4ec8-4380-8f3b-fb4b64ab9ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:18.873153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0a0ea-4ec8-4380-8f3b-fb4b64ab9ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:18.873605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0a0ea-4ec8-4380-8f3b-fb4b64ab9ac8 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:18.874983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0a0ea-4ec8-4380-8f3b-fb4b64ab9ac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:18.875481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0a0ea-4ec8-4380-8f3b-fb4b64ab9ac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:18.877093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0a0ea-4ec8-4380-8f3b-fb4b64ab9ac8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:18.892768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 273/552] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:18.899144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0a0ea-4ec8-4380-8f3b-fb4b64ab9ac8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:19.020940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:19.051424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8f761-fc44-42f5-abab-044fd7d2f40e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:19.064796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0a0ea-4ec8-4380-8f3b-fb4b64ab9ac8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:19.091383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d19db-90d4-48dc-a88a-4fc6b474ed0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:19.092999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d19db-90d4-48dc-a88a-4fc6b474ed0a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:19.093446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d19db-90d4-48dc-a88a-4fc6b474ed0a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:19.093924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d19db-90d4-48dc-a88a-4fc6b474ed0a None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:19.095259 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d19db-90d4-48dc-a88a-4fc6b474ed0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:19.095726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d19db-90d4-48dc-a88a-4fc6b474ed0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:19.097508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d19db-90d4-48dc-a88a-4fc6b474ed0a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:19.099695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:19.115458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 280/553] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:19.115673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d19db-90d4-48dc-a88a-4fc6b474ed0a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:19.137558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9cb0b-9a48-46c1-8357-d14c7085f162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:19.138927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9cb0b-9a48-46c1-8357-d14c7085f162 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:19.139407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9cb0b-9a48-46c1-8357-d14c7085f162 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:19.139970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9cb0b-9a48-46c1-8357-d14c7085f162 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:19.154445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9cb0b-9a48-46c1-8357-d14c7085f162 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:19.157093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9cb0b-9a48-46c1-8357-d14c7085f162 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1ee9cb0b-9a48-46c1-8357-d14c7085f162 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:19.180174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:19.181311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 14:08:19.182300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:19.188834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 274/554] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:19.188834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9cb0b-9a48-46c1-8357-d14c7085f162 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:19.260444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d19db-90d4-48dc-a88a-4fc6b474ed0a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:19.306801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922d275-500c-42fc-b186-5bc931746ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:19.308015 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922d275-500c-42fc-b186-5bc931746ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:19.308531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922d275-500c-42fc-b186-5bc931746ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:19.309098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922d275-500c-42fc-b186-5bc931746ab3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:19.324229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922d275-500c-42fc-b186-5bc931746ab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c922d275-500c-42fc-b186-5bc931746ab3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:19.333659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:19.334687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:19.335516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:19.339034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 281/555] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:19.340196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922d275-500c-42fc-b186-5bc931746ab3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:19.386527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9cb0b-9a48-46c1-8357-d14c7085f162 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:19.413639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:19.455256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac0c4f-0706-4d83-90b0-6e84926af65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:19.456787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac0c4f-0706-4d83-90b0-6e84926af65f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:19.457622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac0c4f-0706-4d83-90b0-6e84926af65f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:19.458415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac0c4f-0706-4d83-90b0-6e84926af65f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:19.471745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922d275-500c-42fc-b186-5bc931746ab3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:19.473914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:19.485278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac0c4f-0706-4d83-90b0-6e84926af65f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:19.488886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac0c4f-0706-4d83-90b0-6e84926af65f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:19.514453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98bc95-8171-4ac8-802d-faf9a45a22de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:19.515163 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98bc95-8171-4ac8-802d-faf9a45a22de None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:19.515476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98bc95-8171-4ac8-802d-faf9a45a22de None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:19.515764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98bc95-8171-4ac8-802d-faf9a45a22de None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:19.516654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98bc95-8171-4ac8-802d-faf9a45a22de None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:19.517176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98bc95-8171-4ac8-802d-faf9a45a22de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:19.518796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98bc95-8171-4ac8-802d-faf9a45a22de None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-41ac0c4f-0706-4d83-90b0-6e84926af65f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:19.529268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:19.530992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:19.533404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:19.533404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:19.533404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:19.533404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:19.537323 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 275/556] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:19.539610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac0c4f-0706-4d83-90b0-6e84926af65f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:19.547600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 282/557] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 6 14:08:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:19.549395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98bc95-8171-4ac8-802d-faf9a45a22de None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:19.683550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac0c4f-0706-4d83-90b0-6e84926af65f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:19.748062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98bc95-8171-4ac8-802d-faf9a45a22de None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:19.758743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b52fc-7b8c-4f1e-adef-e88a1baaf43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:19.760166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b52fc-7b8c-4f1e-adef-e88a1baaf43b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:19.760539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b52fc-7b8c-4f1e-adef-e88a1baaf43b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:19.761042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b52fc-7b8c-4f1e-adef-e88a1baaf43b None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:19.762559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b52fc-7b8c-4f1e-adef-e88a1baaf43b None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:19.763120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b52fc-7b8c-4f1e-adef-e88a1baaf43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:19.765949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b52fc-7b8c-4f1e-adef-e88a1baaf43b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:19.770763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:19.793882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 276/558] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:19.795583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b52fc-7b8c-4f1e-adef-e88a1baaf43b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:19.798316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45f48d-3cc9-4d66-af09-a450e89cf166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:19.799342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45f48d-3cc9-4d66-af09-a450e89cf166 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:19.799873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45f48d-3cc9-4d66-af09-a450e89cf166 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:19.800609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45f48d-3cc9-4d66-af09-a450e89cf166 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:19.801585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45f48d-3cc9-4d66-af09-a450e89cf166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:19.803126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45f48d-3cc9-4d66-af09-a450e89cf166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:19.804846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45f48d-3cc9-4d66-af09-a450e89cf166 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:19.819513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 283/559] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:19.825801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45f48d-3cc9-4d66-af09-a450e89cf166 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:19.898411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:19.973859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45f48d-3cc9-4d66-af09-a450e89cf166 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:20.001212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b52fc-7b8c-4f1e-adef-e88a1baaf43b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:20.012914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea34d25-d3ad-4746-9cf6-71cffc1cf8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:20.013764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea34d25-d3ad-4746-9cf6-71cffc1cf8ef None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:20.014047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea34d25-d3ad-4746-9cf6-71cffc1cf8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:20.014325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea34d25-d3ad-4746-9cf6-71cffc1cf8ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:20.027538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea34d25-d3ad-4746-9cf6-71cffc1cf8ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:20.032221 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea34d25-d3ad-4746-9cf6-71cffc1cf8ef None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-8ea34d25-d3ad-4746-9cf6-71cffc1cf8ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:20.050103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:20.051562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:20.053785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:20.053785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:20.053785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:20.053785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:20.057138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 284/560] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:20.058174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea34d25-d3ad-4746-9cf6-71cffc1cf8ef None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:20.082766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828cee2-1cbf-46eb-a0d1-87d5892cfc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:20.084572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828cee2-1cbf-46eb-a0d1-87d5892cfc1c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:20.085507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828cee2-1cbf-46eb-a0d1-87d5892cfc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:20.086502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828cee2-1cbf-46eb-a0d1-87d5892cfc1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:20.097265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:20.112206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828cee2-1cbf-46eb-a0d1-87d5892cfc1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:20.115306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828cee2-1cbf-46eb-a0d1-87d5892cfc1c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-4828cee2-1cbf-46eb-a0d1-87d5892cfc1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:20.154766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:20.156442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:20.158293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:20.158293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:20.158293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:20.158293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:20.162640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 277/561] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:20.164615 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828cee2-1cbf-46eb-a0d1-87d5892cfc1c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:20.201661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea34d25-d3ad-4746-9cf6-71cffc1cf8ef None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:20.247770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939e30-eeea-4c79-adbb-acb4312afad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:20.248501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939e30-eeea-4c79-adbb-acb4312afad3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:20.248793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939e30-eeea-4c79-adbb-acb4312afad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:20.249075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939e30-eeea-4c79-adbb-acb4312afad3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:20.259903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65939e30-eeea-4c79-adbb-acb4312afad3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:20.261414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65939e30-eeea-4c79-adbb-acb4312afad3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-65939e30-eeea-4c79-adbb-acb4312afad3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:20.280242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:20.282124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:20.284506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:20.286213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 285/562] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:20.287348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939e30-eeea-4c79-adbb-acb4312afad3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:20.381802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828cee2-1cbf-46eb-a0d1-87d5892cfc1c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:20.385383 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:20.422721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:20.445659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939e30-eeea-4c79-adbb-acb4312afad3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:20.448263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91131c0c-6460-4918-aed1-8c1879b373a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:20.450071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91131c0c-6460-4918-aed1-8c1879b373a0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:20.450880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91131c0c-6460-4918-aed1-8c1879b373a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:20.451604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91131c0c-6460-4918-aed1-8c1879b373a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:20.453295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91131c0c-6460-4918-aed1-8c1879b373a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:20.454155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91131c0c-6460-4918-aed1-8c1879b373a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:20.457330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91131c0c-6460-4918-aed1-8c1879b373a0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:20.483071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 278/563] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:20.485315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91131c0c-6460-4918-aed1-8c1879b373a0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:20.496071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f48003-d606-45a7-9e9e-60f4d05bdbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:20.496756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f48003-d606-45a7-9e9e-60f4d05bdbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:20.497045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f48003-d606-45a7-9e9e-60f4d05bdbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:20.497324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f48003-d606-45a7-9e9e-60f4d05bdbf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:20.498180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f48003-d606-45a7-9e9e-60f4d05bdbf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:20.498431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f48003-d606-45a7-9e9e-60f4d05bdbf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:20.499859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f48003-d606-45a7-9e9e-60f4d05bdbf5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:20.523414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 286/564] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:20.526160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7de9df-e1b6-4afa-a552-a00a8ccc0056 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:20.526690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7de9df-e1b6-4afa-a552-a00a8ccc0056 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:20.527155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7de9df-e1b6-4afa-a552-a00a8ccc0056 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:20.529736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 287/565] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:20.535511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7de9df-e1b6-4afa-a552-a00a8ccc0056 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:20.679977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91131c0c-6460-4918-aed1-8c1879b373a0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:20.728369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7de9df-e1b6-4afa-a552-a00a8ccc0056 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:20.744042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782576dd-0b4e-4c66-ad29-df0536cca8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:20.745223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782576dd-0b4e-4c66-ad29-df0536cca8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:20.745758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782576dd-0b4e-4c66-ad29-df0536cca8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:20.746247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782576dd-0b4e-4c66-ad29-df0536cca8f4 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:20.747852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782576dd-0b4e-4c66-ad29-df0536cca8f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:20.748430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782576dd-0b4e-4c66-ad29-df0536cca8f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:20.751135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782576dd-0b4e-4c66-ad29-df0536cca8f4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:20.781133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 279/566] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 6 14:08:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:20.782710 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782576dd-0b4e-4c66-ad29-df0536cca8f4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:20.799180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f0c89-799e-4a2a-9afa-eb07b9e4738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:20.800402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f0c89-799e-4a2a-9afa-eb07b9e4738c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:20.800886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f0c89-799e-4a2a-9afa-eb07b9e4738c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:20.801338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f0c89-799e-4a2a-9afa-eb07b9e4738c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:20.814571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f0c89-799e-4a2a-9afa-eb07b9e4738c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:20.816785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f0c89-799e-4a2a-9afa-eb07b9e4738c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:20.829429 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-e34f0c89-799e-4a2a-9afa-eb07b9e4738c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:20.843574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:20.845429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 288/567] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 6 14:08:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f08084-3b1a-47c5-a3ae-8e35cb14ae18 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:20.847953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f08084-3b1a-47c5-a3ae-8e35cb14ae18 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:20.849305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f08084-3b1a-47c5-a3ae-8e35cb14ae18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:20.871002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:20.881682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a6f08084-3b1a-47c5-a3ae-8e35cb14ae18 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:20.883377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a6f08084-3b1a-47c5-a3ae-8e35cb14ae18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:20.964406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782576dd-0b4e-4c66-ad29-df0536cca8f4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:20.973838 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 289/568] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:20.974689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f08084-3b1a-47c5-a3ae-8e35cb14ae18 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:21.033120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fcf8-c728-41c6-8055-31b362aac77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:21.034377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fcf8-c728-41c6-8055-31b362aac77c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.034820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fcf8-c728-41c6-8055-31b362aac77c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.035268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fcf8-c728-41c6-8055-31b362aac77c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.048179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1fcf8-c728-41c6-8055-31b362aac77c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:21.050761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1fcf8-c728-41c6-8055-31b362aac77c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-efb1fcf8-c728-41c6-8055-31b362aac77c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:21.072765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:21.074773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 280/569] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:21.076566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fcf8-c728-41c6-8055-31b362aac77c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:21.103377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f08084-3b1a-47c5-a3ae-8e35cb14ae18 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:21.145167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920a7a0-e130-4751-8538-4e450d3711ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:21.145951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920a7a0-e130-4751-8538-4e450d3711ff None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.146242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920a7a0-e130-4751-8538-4e450d3711ff None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.146524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920a7a0-e130-4751-8538-4e450d3711ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.151446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:21.191495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:21.224299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fcf8-c728-41c6-8055-31b362aac77c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:21.247115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920a7a0-e130-4751-8538-4e450d3711ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:21.248850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920a7a0-e130-4751-8538-4e450d3711ff None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:21.278282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd72e1-e1ee-4e06-9841-ef1f8af52063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:21.278962 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd72e1-e1ee-4e06-9841-ef1f8af52063 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.279243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd72e1-e1ee-4e06-9841-ef1f8af52063 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.279515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd72e1-e1ee-4e06-9841-ef1f8af52063 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.280399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd72e1-e1ee-4e06-9841-ef1f8af52063 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:21.280704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd72e1-e1ee-4e06-9841-ef1f8af52063 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:21.282240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd72e1-e1ee-4e06-9841-ef1f8af52063 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:21.308308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 281/570] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 6 14:08:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:21.310678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a4676-c6d4-4be9-aece-b0db6d018bf3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.311177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a4676-c6d4-4be9-aece-b0db6d018bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.311684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a4676-c6d4-4be9-aece-b0db6d018bf3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.313996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 282/571] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:21.315223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a4676-c6d4-4be9-aece-b0db6d018bf3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:21.389051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 290/572] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:21.390464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920a7a0-e130-4751-8538-4e450d3711ff None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:21.442690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:21.451479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a4676-c6d4-4be9-aece-b0db6d018bf3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:21.491891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a868c-946a-4f12-baf8-c2f74ca2802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:21.492697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a868c-946a-4f12-baf8-c2f74ca2802a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.492978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a868c-946a-4f12-baf8-c2f74ca2802a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.493554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a868c-946a-4f12-baf8-c2f74ca2802a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.506410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:21.512952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a868c-946a-4f12-baf8-c2f74ca2802a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f67a868c-946a-4f12-baf8-c2f74ca2802a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:21.529359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:21.536629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 283/573] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:21.538321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a868c-946a-4f12-baf8-c2f74ca2802a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:21.558046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920a7a0-e130-4751-8538-4e450d3711ff None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:21.607869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56c564-212f-4bb4-95af-4b91b4d19eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:21.609086 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56c564-212f-4bb4-95af-4b91b4d19eba None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.609546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56c564-212f-4bb4-95af-4b91b4d19eba None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.610015 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56c564-212f-4bb4-95af-4b91b4d19eba None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.611228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56c564-212f-4bb4-95af-4b91b4d19eba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:21.611740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56c564-212f-4bb4-95af-4b91b4d19eba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:21.614537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56c564-212f-4bb4-95af-4b91b4d19eba None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:21.635144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 291/574] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:21.636882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b815dfc-230f-4ca4-abbe-d381afbedf8a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.637341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b815dfc-230f-4ca4-abbe-d381afbedf8a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.637838 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b815dfc-230f-4ca4-abbe-d381afbedf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.660478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8b815dfc-230f-4ca4-abbe-d381afbedf8a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:21.661886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8b815dfc-230f-4ca4-abbe-d381afbedf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:21.679263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a868c-946a-4f12-baf8-c2f74ca2802a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:21.720103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dfad4-aace-4318-92be-69b414b13862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:21.720989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dfad4-aace-4318-92be-69b414b13862 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.721397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dfad4-aace-4318-92be-69b414b13862 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.721794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dfad4-aace-4318-92be-69b414b13862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.734420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dfad4-aace-4318-92be-69b414b13862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:21.735904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dfad4-aace-4318-92be-69b414b13862 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a80dfad4-aace-4318-92be-69b414b13862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:21.756828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:21.758930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:21.761620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:21.761620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:21.761620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:21.761620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:21.763067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 284/575] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:21.763860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dfad4-aace-4318-92be-69b414b13862 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:21.780189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 292/576] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:21.781636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b815dfc-230f-4ca4-abbe-d381afbedf8a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:21.832577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:21.833550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:21.908298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dfad4-aace-4318-92be-69b414b13862 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:21.943083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b815dfc-230f-4ca4-abbe-d381afbedf8a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:21.959316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7231f0a-ce35-4846-aadd-df74210e6dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:21.960369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7231f0a-ce35-4846-aadd-df74210e6dff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.960752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7231f0a-ce35-4846-aadd-df74210e6dff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.961118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7231f0a-ce35-4846-aadd-df74210e6dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.975858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7231f0a-ce35-4846-aadd-df74210e6dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:21.977683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7231f0a-ce35-4846-aadd-df74210e6dff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:21.991664 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b890452-5909-417b-aef6-5c3921f75540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:21.992440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b890452-5909-417b-aef6-5c3921f75540 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:21.992794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b890452-5909-417b-aef6-5c3921f75540 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:21.993647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b890452-5909-417b-aef6-5c3921f75540 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:21.995013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b890452-5909-417b-aef6-5c3921f75540 None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:21.995562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b890452-5909-417b-aef6-5c3921f75540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:21.997697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b890452-5909-417b-aef6-5c3921f75540 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f7231f0a-ce35-4846-aadd-df74210e6dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:21.999872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:22.001745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:22.003033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:22.003033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:22.003033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:22.003033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:22.003033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 285/577] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:22.003033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7231f0a-ce35-4846-aadd-df74210e6dff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:22.020608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 293/578] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:22.020608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b890452-5909-417b-aef6-5c3921f75540 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:22.137387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:22.154133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b890452-5909-417b-aef6-5c3921f75540 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:22.162100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7231f0a-ce35-4846-aadd-df74210e6dff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:22.167132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:22.194027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46de702-3277-4523-8111-79eb3338078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:22.195139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de702-3277-4523-8111-79eb3338078f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:22.195570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de702-3277-4523-8111-79eb3338078f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:22.196011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de702-3277-4523-8111-79eb3338078f None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:22.197348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46de702-3277-4523-8111-79eb3338078f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:22.197738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46de702-3277-4523-8111-79eb3338078f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:22.199168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46de702-3277-4523-8111-79eb3338078f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:22.212223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 294/579] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:22.213106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46de702-3277-4523-8111-79eb3338078f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:22.228995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c1931-63da-41d8-b534-aadd18cd3dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:22.230842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c1931-63da-41d8-b534-aadd18cd3dd8 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:22.231291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c1931-63da-41d8-b534-aadd18cd3dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:22.231737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c1931-63da-41d8-b534-aadd18cd3dd8 None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/f41a357957bc47739b7bd6f21d159d74/roles/4107928942534d3984671dc5232d061d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:22.256354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c1931-63da-41d8-b534-aadd18cd3dd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=f41a357957bc47739b7bd6f21d159d74, role_id=4107928942534d3984671dc5232d061d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:22.259373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c1931-63da-41d8-b534-aadd18cd3dd8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:22.278893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 286/580] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:08:22 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/f41a357957bc47739b7bd6f21d159d74/roles/4107928942534d3984671dc5232d061d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:08:22.280090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c1931-63da-41d8-b534-aadd18cd3dd8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:22.382939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46de702-3277-4523-8111-79eb3338078f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:22.429813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c1931-63da-41d8-b534-aadd18cd3dd8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:22.446571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72dc21-0311-49e5-9752-55f636bb2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:22.447911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72dc21-0311-49e5-9752-55f636bb2fbe None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:22.448379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72dc21-0311-49e5-9752-55f636bb2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:22.448845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72dc21-0311-49e5-9752-55f636bb2fbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:22.469010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72dc21-0311-49e5-9752-55f636bb2fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:22.479559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f912b0-ca18-4e86-b403-533f6e90a93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:22.480433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f912b0-ca18-4e86-b403-533f6e90a93d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:22.480843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f912b0-ca18-4e86-b403-533f6e90a93d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:22.481268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f912b0-ca18-4e86-b403-533f6e90a93d None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:22.482238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f912b0-ca18-4e86-b403-533f6e90a93d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:22.482682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f912b0-ca18-4e86-b403-533f6e90a93d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-9c72dc21-0311-49e5-9752-55f636bb2fbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:22.483753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:22.486296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:22.487898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:22.490584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f912b0-ca18-4e86-b403-533f6e90a93d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:22.490826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 295/581] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:22.490826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72dc21-0311-49e5-9752-55f636bb2fbe None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:22.510815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:22.521288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 287/582] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:22.521288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f912b0-ca18-4e86-b403-533f6e90a93d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:22.551146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:22.659966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72dc21-0311-49e5-9752-55f636bb2fbe None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:22.685147 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f912b0-ca18-4e86-b403-533f6e90a93d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:22.719319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c17ffb-f891-4264-a98b-84827f3838b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:22.720452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c17ffb-f891-4264-a98b-84827f3838b0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:22.721001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c17ffb-f891-4264-a98b-84827f3838b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:22.721489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c17ffb-f891-4264-a98b-84827f3838b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:22.724041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b93a7-bdfc-481e-a615-1eb1364839f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:22.724880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b93a7-bdfc-481e-a615-1eb1364839f1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:22.725156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b93a7-bdfc-481e-a615-1eb1364839f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:22.725433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b93a7-bdfc-481e-a615-1eb1364839f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:22.733434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b93a7-bdfc-481e-a615-1eb1364839f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:22.736784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b93a7-bdfc-481e-a615-1eb1364839f1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:22.747698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c17ffb-f891-4264-a98b-84827f3838b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:22.753318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c17ffb-f891-4264-a98b-84827f3838b0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-106b93a7-bdfc-481e-a615-1eb1364839f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:22.763284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:22.764600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:22.765502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:22.765502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:22.765502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:22.765502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:22.765502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:22.765502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:22.765502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:22.765502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:22.765502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 288/583] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:22.767299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b93a7-bdfc-481e-a615-1eb1364839f1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-71c17ffb-f891-4264-a98b-84827f3838b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:22.783172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:22.785233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:22.787374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:22.787374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:22.787374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:22.787374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:22.788567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 296/584] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:22.791860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c17ffb-f891-4264-a98b-84827f3838b0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:22.860409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:22.891672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b93a7-bdfc-481e-a615-1eb1364839f1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:22.899561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:22.931755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d9c37-06ff-4984-8ceb-e258cbf60781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:22.932493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d9c37-06ff-4984-8ceb-e258cbf60781 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:22.932785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d9c37-06ff-4984-8ceb-e258cbf60781 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:22.933191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d9c37-06ff-4984-8ceb-e258cbf60781 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:22.933988 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d9c37-06ff-4984-8ceb-e258cbf60781 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:22.934300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d9c37-06ff-4984-8ceb-e258cbf60781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:22.935902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d9c37-06ff-4984-8ceb-e258cbf60781 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:22.945507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 289/585] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:22.947268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d9c37-06ff-4984-8ceb-e258cbf60781 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:22.980582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c17ffb-f891-4264-a98b-84827f3838b0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:23.030844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307f855-9e73-44a8-a008-05ad807d7c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:23.031762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307f855-9e73-44a8-a008-05ad807d7c0f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:23.032053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307f855-9e73-44a8-a008-05ad807d7c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:23.032391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307f855-9e73-44a8-a008-05ad807d7c0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:23.041237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307f855-9e73-44a8-a008-05ad807d7c0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:23.043501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307f855-9e73-44a8-a008-05ad807d7c0f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-4307f855-9e73-44a8-a008-05ad807d7c0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:23.060167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:23.061219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:23.062179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:23.062179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:23.062179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:23.062179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:23.062179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:23.062179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:23.062179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:23.062179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:23.062179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 297/586] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:23.062676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307f855-9e73-44a8-a008-05ad807d7c0f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:23.077937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d9c37-06ff-4984-8ceb-e258cbf60781 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:23.116525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa391-7a70-42dc-b8d2-2f01909b3b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:23.117191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa391-7a70-42dc-b8d2-2f01909b3b07 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:23.117470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa391-7a70-42dc-b8d2-2f01909b3b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:23.117866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa391-7a70-42dc-b8d2-2f01909b3b07 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:23.118819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa391-7a70-42dc-b8d2-2f01909b3b07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:23.120532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa391-7a70-42dc-b8d2-2f01909b3b07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:23.122128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa391-7a70-42dc-b8d2-2f01909b3b07 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:23.126703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:23.145520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 290/587] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:23.146292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa391-7a70-42dc-b8d2-2f01909b3b07 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:23.281184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307f855-9e73-44a8-a008-05ad807d7c0f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:23.304672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:23.324262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c07c9-11a9-45ae-a2a2-7bc044df2cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:23.325445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c07c9-11a9-45ae-a2a2-7bc044df2cfb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:23.325732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa391-7a70-42dc-b8d2-2f01909b3b07 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:23.326024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c07c9-11a9-45ae-a2a2-7bc044df2cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:23.326509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c07c9-11a9-45ae-a2a2-7bc044df2cfb None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:23.327907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c07c9-11a9-45ae-a2a2-7bc044df2cfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:23.328463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c07c9-11a9-45ae-a2a2-7bc044df2cfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:23.331101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c07c9-11a9-45ae-a2a2-7bc044df2cfb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:23.354977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 298/588] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:23.354977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c07c9-11a9-45ae-a2a2-7bc044df2cfb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:23.395451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e732915-3011-4d51-bbc1-4cae3f7466de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:23.396767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e732915-3011-4d51-bbc1-4cae3f7466de None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:23.397384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e732915-3011-4d51-bbc1-4cae3f7466de None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:23.397913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e732915-3011-4d51-bbc1-4cae3f7466de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:23.418718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e732915-3011-4d51-bbc1-4cae3f7466de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:23.421249 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e732915-3011-4d51-bbc1-4cae3f7466de None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1e732915-3011-4d51-bbc1-4cae3f7466de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:23.449942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:23.452057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:23.453980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:23.453980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:23.453980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:23.453980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:23.453980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 291/589] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:23.454984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e732915-3011-4d51-bbc1-4cae3f7466de None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:23.529630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c07c9-11a9-45ae-a2a2-7bc044df2cfb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:23.541500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:23.602043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307c68a-89ff-4da3-8ecd-6500cc3f9476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:23.603530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307c68a-89ff-4da3-8ecd-6500cc3f9476 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:23.603972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307c68a-89ff-4da3-8ecd-6500cc3f9476 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:23.604428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307c68a-89ff-4da3-8ecd-6500cc3f9476 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:23.605487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307c68a-89ff-4da3-8ecd-6500cc3f9476 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:23.605984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307c68a-89ff-4da3-8ecd-6500cc3f9476 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:23.607787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307c68a-89ff-4da3-8ecd-6500cc3f9476 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:23.634271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 299/590] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:23.634271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307c68a-89ff-4da3-8ecd-6500cc3f9476 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:23.653167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e732915-3011-4d51-bbc1-4cae3f7466de None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:23.698820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:23.711192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abc7af-be8b-4f54-b460-41b89f0353c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:23.712528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc7af-be8b-4f54-b460-41b89f0353c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:23.712938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc7af-be8b-4f54-b460-41b89f0353c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:23.713355 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc7af-be8b-4f54-b460-41b89f0353c7 None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/0b82a920df4042dd97314350aed4d7a5/roles/4107928942534d3984671dc5232d061d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:23.732431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abc7af-be8b-4f54-b460-41b89f0353c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=0b82a920df4042dd97314350aed4d7a5, role_id=4107928942534d3984671dc5232d061d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:23.735136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abc7af-be8b-4f54-b460-41b89f0353c7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:23.761748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 292/591] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:08:23 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/0b82a920df4042dd97314350aed4d7a5/roles/4107928942534d3984671dc5232d061d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:08:23.761748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abc7af-be8b-4f54-b460-41b89f0353c7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:23.836769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307c68a-89ff-4da3-8ecd-6500cc3f9476 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:23.919358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3da0ce-20b8-4f66-9c8a-70aebbf96558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:23.920898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3da0ce-20b8-4f66-9c8a-70aebbf96558 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:23.921474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3da0ce-20b8-4f66-9c8a-70aebbf96558 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:23.922129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3da0ce-20b8-4f66-9c8a-70aebbf96558 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:23.935848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3da0ce-20b8-4f66-9c8a-70aebbf96558 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:23.938717 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3da0ce-20b8-4f66-9c8a-70aebbf96558 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:23.950102 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:23.962511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abc7af-be8b-4f54-b460-41b89f0353c7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2e3da0ce-20b8-4f66-9c8a-70aebbf96558 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:23.966945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:23.969097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 14:08:23.971273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:23.973507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 300/592] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:23.974464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3da0ce-20b8-4f66-9c8a-70aebbf96558 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:24.045072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdab0e-30e3-4d37-899d-64f8cf67ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:24.046585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdab0e-30e3-4d37-899d-64f8cf67ce3d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:24.047277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdab0e-30e3-4d37-899d-64f8cf67ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:24.048004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdab0e-30e3-4d37-899d-64f8cf67ce3d None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:24.049612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdab0e-30e3-4d37-899d-64f8cf67ce3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:24.050413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdab0e-30e3-4d37-899d-64f8cf67ce3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:24.053025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdab0e-30e3-4d37-899d-64f8cf67ce3d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:24.078064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 293/593] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:24.079896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdab0e-30e3-4d37-899d-64f8cf67ce3d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:24.183444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:24.191719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3da0ce-20b8-4f66-9c8a-70aebbf96558 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:24.238303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844c5fa-ac1e-46d1-b611-84d81f3675c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:24.239342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844c5fa-ac1e-46d1-b611-84d81f3675c4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:24.239785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844c5fa-ac1e-46d1-b611-84d81f3675c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:24.240247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844c5fa-ac1e-46d1-b611-84d81f3675c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:24.241366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844c5fa-ac1e-46d1-b611-84d81f3675c4 None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=3571f3d3a0de4e9abe55ed84426143a0) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:24.241902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844c5fa-ac1e-46d1-b611-84d81f3675c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:24.244000 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844c5fa-ac1e-46d1-b611-84d81f3675c4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:24.246646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdab0e-30e3-4d37-899d-64f8cf67ce3d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:24.252045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 301/594] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:23 2022] GET /identity/v3/role_assignments?user.id=3571f3d3a0de4e9abe55ed84426143a0&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:24.253267 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844c5fa-ac1e-46d1-b611-84d81f3675c4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:24.320339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d6efd-cc85-4b67-9af1-a5503f51f34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:24.322052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d6efd-cc85-4b67-9af1-a5503f51f34c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:24.322781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d6efd-cc85-4b67-9af1-a5503f51f34c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:24.323305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d6efd-cc85-4b67-9af1-a5503f51f34c None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:24.325243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d6efd-cc85-4b67-9af1-a5503f51f34c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:24.326221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d6efd-cc85-4b67-9af1-a5503f51f34c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:24.329273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d6efd-cc85-4b67-9af1-a5503f51f34c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:24.368396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 294/595] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 6 14:08:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:24.377011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d6efd-cc85-4b67-9af1-a5503f51f34c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:24.384589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844c5fa-ac1e-46d1-b611-84d81f3675c4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:24.429782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4544876-7465-44ef-83a3-62d8713dbe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:24.431093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4544876-7465-44ef-83a3-62d8713dbe7c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:24.431581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4544876-7465-44ef-83a3-62d8713dbe7c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:24.432086 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4544876-7465-44ef-83a3-62d8713dbe7c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:24.441679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:24.443198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4544876-7465-44ef-83a3-62d8713dbe7c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:24.445338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4544876-7465-44ef-83a3-62d8713dbe7c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a4544876-7465-44ef-83a3-62d8713dbe7c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:24.464210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:24.465304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 14:08:24.466317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:24.469281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 302/596] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:24.478935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4544876-7465-44ef-83a3-62d8713dbe7c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:24.519234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:24.615408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4544876-7465-44ef-83a3-62d8713dbe7c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:24.657615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe2146-b0d6-4e23-8c86-3904bb2e049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:24.658795 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe2146-b0d6-4e23-8c86-3904bb2e049b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:24.659308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe2146-b0d6-4e23-8c86-3904bb2e049b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:24.659802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe2146-b0d6-4e23-8c86-3904bb2e049b None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:24.661023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe2146-b0d6-4e23-8c86-3904bb2e049b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:24.661623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe2146-b0d6-4e23-8c86-3904bb2e049b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:24.663419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe2146-b0d6-4e23-8c86-3904bb2e049b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:24.687798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 303/597] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Jun 6 14:08:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:24.697782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe2146-b0d6-4e23-8c86-3904bb2e049b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:24.725772 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d6efd-cc85-4b67-9af1-a5503f51f34c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:24.814853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df0d13-ea12-4219-9788-b08d199f01be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:24.816241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0d13-ea12-4219-9788-b08d199f01be None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:24.816749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0d13-ea12-4219-9788-b08d199f01be None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:24.817240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0d13-ea12-4219-9788-b08d199f01be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:24.839835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df0d13-ea12-4219-9788-b08d199f01be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:24.842550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df0d13-ea12-4219-9788-b08d199f01be None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a4df0d13-ea12-4219-9788-b08d199f01be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:24.874989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:24.875981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:24.876990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:24.876990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:24.876990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:24.876990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:24.878331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 295/598] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 503 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:24.887801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df0d13-ea12-4219-9788-b08d199f01be None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:24.998191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:25.058193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe2146-b0d6-4e23-8c86-3904bb2e049b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:25.119231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:25.154581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4096fb-8f17-4718-8b1b-bc32cd50a518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:25.155416 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4096fb-8f17-4718-8b1b-bc32cd50a518 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:25.155701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4096fb-8f17-4718-8b1b-bc32cd50a518 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:25.155975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4096fb-8f17-4718-8b1b-bc32cd50a518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:25.172824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4096fb-8f17-4718-8b1b-bc32cd50a518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:25.174758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4096fb-8f17-4718-8b1b-bc32cd50a518 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:25.188916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df0d13-ea12-4219-9788-b08d199f01be None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-fd4096fb-8f17-4718-8b1b-bc32cd50a518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:25.196873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:25.198213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:25.199148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:25.199148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:25.199148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:25.199148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:25.201699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 304/599] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 505 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:25.203384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3036bef-7d12-46e9-af33-45de85116661 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:25.203911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3036bef-7d12-46e9-af33-45de85116661 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:25.204402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3036bef-7d12-46e9-af33-45de85116661 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:25.210497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 305/600] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:25.211767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3036bef-7d12-46e9-af33-45de85116661 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:25.266725 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87554dfd-1d7a-4c94-b9fa-3517deead979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:25.268187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87554dfd-1d7a-4c94-b9fa-3517deead979 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:25.268487 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87554dfd-1d7a-4c94-b9fa-3517deead979 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:25.268764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87554dfd-1d7a-4c94-b9fa-3517deead979 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:25.269860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87554dfd-1d7a-4c94-b9fa-3517deead979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:25.270170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87554dfd-1d7a-4c94-b9fa-3517deead979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:25.271938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87554dfd-1d7a-4c94-b9fa-3517deead979 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:25.295664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 296/601] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:25.298294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ee869-b7fc-440e-a3d1-d37661424f90 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:25.299145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ee869-b7fc-440e-a3d1-d37661424f90 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:25.299889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ee869-b7fc-440e-a3d1-d37661424f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:25.323512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5c3ee869-b7fc-440e-a3d1-d37661424f90 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:25.325116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5c3ee869-b7fc-440e-a3d1-d37661424f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:25.363717 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3036bef-7d12-46e9-af33-45de85116661 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:25.417872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 297/602] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:25.419265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ee869-b7fc-440e-a3d1-d37661424f90 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:25.442570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409ce1f-4c18-4f90-ad69-139f7990f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:25.444078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409ce1f-4c18-4f90-ad69-139f7990f2fb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:25.445102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409ce1f-4c18-4f90-ad69-139f7990f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:25.445585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409ce1f-4c18-4f90-ad69-139f7990f2fb None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:25.448005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409ce1f-4c18-4f90-ad69-139f7990f2fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:25.448596 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409ce1f-4c18-4f90-ad69-139f7990f2fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:25.452022 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409ce1f-4c18-4f90-ad69-139f7990f2fb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:25.456564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:25.482447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 306/603] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:25.484587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409ce1f-4c18-4f90-ad69-139f7990f2fb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:25.541000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:25.628615 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ee869-b7fc-440e-a3d1-d37661424f90 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:25.669926 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0e552-fb63-4487-bd2d-d7550ae27060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:25.671138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0e552-fb63-4487-bd2d-d7550ae27060 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:25.671638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0e552-fb63-4487-bd2d-d7550ae27060 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:25.672156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0e552-fb63-4487-bd2d-d7550ae27060 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:25.681318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0e552-fb63-4487-bd2d-d7550ae27060 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:25.683314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0e552-fb63-4487-bd2d-d7550ae27060 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:25.685162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409ce1f-4c18-4f90-ad69-139f7990f2fb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c9d0e552-fb63-4487-bd2d-d7550ae27060 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:25.700593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:25.701571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:25.702521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:25.702521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:25.702521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:25.702521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:25.702521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:25.702521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:25.702521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:25.702521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:25.709589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 298/604] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:25.710917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0e552-fb63-4487-bd2d-d7550ae27060 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:25.755063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406a1c4-5f59-4c86-ac87-2f17d5c39cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:25.756592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406a1c4-5f59-4c86-ac87-2f17d5c39cdd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:25.757192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406a1c4-5f59-4c86-ac87-2f17d5c39cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:25.757736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406a1c4-5f59-4c86-ac87-2f17d5c39cdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:25.778912 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406a1c4-5f59-4c86-ac87-2f17d5c39cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-3406a1c4-5f59-4c86-ac87-2f17d5c39cdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:25.794621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:25.795613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:25.796463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:25.802154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 307/605] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:25.803841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406a1c4-5f59-4c86-ac87-2f17d5c39cdd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:25.833699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0e552-fb63-4487-bd2d-d7550ae27060 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:25.853916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:25.874957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543b645-2543-4140-bda4-c471f63ba3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:25.876143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543b645-2543-4140-bda4-c471f63ba3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:25.876635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543b645-2543-4140-bda4-c471f63ba3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:25.877141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543b645-2543-4140-bda4-c471f63ba3a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:25.883187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:25.886250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543b645-2543-4140-bda4-c471f63ba3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:25.888393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543b645-2543-4140-bda4-c471f63ba3a9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9543b645-2543-4140-bda4-c471f63ba3a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:25.904211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:25.905214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:25.906146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:25.906146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:25.906146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:25.906146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:25.906146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:25.906146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:25.906146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:25.906146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:25.908718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 299/606] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:25.909910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543b645-2543-4140-bda4-c471f63ba3a9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:25.941359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406a1c4-5f59-4c86-ac87-2f17d5c39cdd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:25.980428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa4109-c7c4-479e-b960-96090d327fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:25.981517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa4109-c7c4-479e-b960-96090d327fcc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:25.982036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa4109-c7c4-479e-b960-96090d327fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:25.982507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa4109-c7c4-479e-b960-96090d327fcc None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:25.984589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa4109-c7c4-479e-b960-96090d327fcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:25.984732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa4109-c7c4-479e-b960-96090d327fcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:25.987644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa4109-c7c4-479e-b960-96090d327fcc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:26.005894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 308/607] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:26.005894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa4109-c7c4-479e-b960-96090d327fcc None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:26.076447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543b645-2543-4140-bda4-c471f63ba3a9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:26.117687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b15b3f-9b85-4d21-b32b-4b5949886239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:26.119113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b15b3f-9b85-4d21-b32b-4b5949886239 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:26.119497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b15b3f-9b85-4d21-b32b-4b5949886239 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:26.120087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b15b3f-9b85-4d21-b32b-4b5949886239 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:26.120730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b15b3f-9b85-4d21-b32b-4b5949886239 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:26.121126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b15b3f-9b85-4d21-b32b-4b5949886239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:26.122637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b15b3f-9b85-4d21-b32b-4b5949886239 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:26.127348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa4109-c7c4-479e-b960-96090d327fcc None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:26.136442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 300/608] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:26.136442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b15b3f-9b85-4d21-b32b-4b5949886239 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:26.149198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:26.175222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573eac8-08e1-485a-bd99-f922d31bc322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:26.176177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573eac8-08e1-485a-bd99-f922d31bc322 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:26.176614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573eac8-08e1-485a-bd99-f922d31bc322 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:26.177054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573eac8-08e1-485a-bd99-f922d31bc322 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:26.178050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573eac8-08e1-485a-bd99-f922d31bc322 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:26.178515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573eac8-08e1-485a-bd99-f922d31bc322 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:26.180009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:26.180328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573eac8-08e1-485a-bd99-f922d31bc322 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:26.199445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 309/609] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:26.200782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573eac8-08e1-485a-bd99-f922d31bc322 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:26.248452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b15b3f-9b85-4d21-b32b-4b5949886239 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:26.285949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc51ab7-890f-46f7-90c8-a71d2ef56aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:26.286859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc51ab7-890f-46f7-90c8-a71d2ef56aa3 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:26.287521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc51ab7-890f-46f7-90c8-a71d2ef56aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:26.287521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc51ab7-890f-46f7-90c8-a71d2ef56aa3 None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/720c995318a24e7b859fa82febd8178b/roles/4107928942534d3984671dc5232d061d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:26.304140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc51ab7-890f-46f7-90c8-a71d2ef56aa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=720c995318a24e7b859fa82febd8178b, role_id=4107928942534d3984671dc5232d061d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:26.306143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc51ab7-890f-46f7-90c8-a71d2ef56aa3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:26.318813 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573eac8-08e1-485a-bd99-f922d31bc322 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:26.328912 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 301/610] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:08:26 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/720c995318a24e7b859fa82febd8178b/roles/4107928942534d3984671dc5232d061d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:08:26.338638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc51ab7-890f-46f7-90c8-a71d2ef56aa3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:26.363776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd16a4f-8a38-4e0a-b973-c980451ccc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:26.365010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd16a4f-8a38-4e0a-b973-c980451ccc84 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:26.365505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd16a4f-8a38-4e0a-b973-c980451ccc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:26.366029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd16a4f-8a38-4e0a-b973-c980451ccc84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:26.381308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd16a4f-8a38-4e0a-b973-c980451ccc84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:26.383850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd16a4f-8a38-4e0a-b973-c980451ccc84 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-bfd16a4f-8a38-4e0a-b973-c980451ccc84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:26.422919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:26.423925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:26.424837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:26.424837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:26.424837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:26.424837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:26.428201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 310/611] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:26.433677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd16a4f-8a38-4e0a-b973-c980451ccc84 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:26.492670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:26.519515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:26.543106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc51ab7-890f-46f7-90c8-a71d2ef56aa3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:26.626911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c87c8-6749-47fe-97bf-eea782386f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:26.627750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c87c8-6749-47fe-97bf-eea782386f17 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:26.628035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c87c8-6749-47fe-97bf-eea782386f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:26.628314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c87c8-6749-47fe-97bf-eea782386f17 None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/59cd9d922204496d8ea185716b832481/roles/4107928942534d3984671dc5232d061d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:26.662682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c87c8-6749-47fe-97bf-eea782386f17 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=59cd9d922204496d8ea185716b832481, role_id=4107928942534d3984671dc5232d061d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:26.664185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c87c8-6749-47fe-97bf-eea782386f17 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:26.672301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd16a4f-8a38-4e0a-b973-c980451ccc84 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:26.693414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 302/612] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:08:26 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/59cd9d922204496d8ea185716b832481/roles/4107928942534d3984671dc5232d061d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:08:26.744469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b331e-bd34-4e7c-b804-4190a0dfb33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:26.745253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b331e-bd34-4e7c-b804-4190a0dfb33b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:26.745422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b331e-bd34-4e7c-b804-4190a0dfb33b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:26.745671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b331e-bd34-4e7c-b804-4190a0dfb33b None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:26.746463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b331e-bd34-4e7c-b804-4190a0dfb33b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:26.746772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b331e-bd34-4e7c-b804-4190a0dfb33b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:26.748223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b331e-bd34-4e7c-b804-4190a0dfb33b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:26.771049 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 311/613] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:26.777361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c87c8-6749-47fe-97bf-eea782386f17 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:26.949984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c87c8-6749-47fe-97bf-eea782386f17 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:26.990625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b98b5f-5429-4c86-9a24-82c08328dc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:26.991449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b98b5f-5429-4c86-9a24-82c08328dc76 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:26.991735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b98b5f-5429-4c86-9a24-82c08328dc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:26.992015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b98b5f-5429-4c86-9a24-82c08328dc76 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.001191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b040a-8258-43cd-8adb-5bfd5cb6fd1b None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.001456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b040a-8258-43cd-8adb-5bfd5cb6fd1b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.001764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b040a-8258-43cd-8adb-5bfd5cb6fd1b None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.003276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b98b5f-5429-4c86-9a24-82c08328dc76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:27.003440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 312/614] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:27.005197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b98b5f-5429-4c86-9a24-82c08328dc76 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:27.009176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:27.009448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52953671-677b-4ad2-af47-ff58ea1cc0b2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.009697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52953671-677b-4ad2-af47-ff58ea1cc0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.010702 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52953671-677b-4ad2-af47-ff58ea1cc0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.039643 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-52953671-677b-4ad2-af47-ff58ea1cc0b2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:27.040857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-52953671-677b-4ad2-af47-ff58ea1cc0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-b3b98b5f-5429-4c86-9a24-82c08328dc76 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:27.049826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:27.052158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 14:08:27.054136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:27.060957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 303/615] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:27.065195 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b98b5f-5429-4c86-9a24-82c08328dc76 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:27.127894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 313/616] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 06 14:08:27.144260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b67a-94f6-40ae-a2b9-c61382c66d05 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.144569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b67a-94f6-40ae-a2b9-c61382c66d05 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.144808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b67a-94f6-40ae-a2b9-c61382c66d05 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.146615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 314/617] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:27.156947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a6bdc-8dbf-4736-a653-b0572c9b32d2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.157233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a6bdc-8dbf-4736-a653-b0572c9b32d2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.157506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a6bdc-8dbf-4736-a653-b0572c9b32d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.175801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-813a6bdc-8dbf-4736-a653-b0572c9b32d2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:27.177009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-813a6bdc-8dbf-4736-a653-b0572c9b32d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:27.190712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:27.241402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b98b5f-5429-4c86-9a24-82c08328dc76 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:27.258730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 315/618] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:27.260162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a6bdc-8dbf-4736-a653-b0572c9b32d2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:27.284368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f19b83-4c40-4fe7-80d5-6d322898aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:27.285388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f19b83-4c40-4fe7-80d5-6d322898aa0a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.285921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f19b83-4c40-4fe7-80d5-6d322898aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.286552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f19b83-4c40-4fe7-80d5-6d322898aa0a None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.287628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f19b83-4c40-4fe7-80d5-6d322898aa0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:27.288186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f19b83-4c40-4fe7-80d5-6d322898aa0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:27.290208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f19b83-4c40-4fe7-80d5-6d322898aa0a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:27.315991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 304/619] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 6 14:08:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:27.317035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f19b83-4c40-4fe7-80d5-6d322898aa0a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:27.378709 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a6bdc-8dbf-4736-a653-b0572c9b32d2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:27.394362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:27.438600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50659e-0412-46d2-9c20-211fc879b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:27.439955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50659e-0412-46d2-9c20-211fc879b69c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.440454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50659e-0412-46d2-9c20-211fc879b69c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.440886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50659e-0412-46d2-9c20-211fc879b69c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.466879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50659e-0412-46d2-9c20-211fc879b69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:27.473745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f19b83-4c40-4fe7-80d5-6d322898aa0a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-4c50659e-0412-46d2-9c20-211fc879b69c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:27.481182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:27.482537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:27.483584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:27.494053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 316/620] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:27.495365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50659e-0412-46d2-9c20-211fc879b69c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:27.511219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:27.548668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e7f0-b9af-4b3c-b794-52502bd9f554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:27.549841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e7f0-b9af-4b3c-b794-52502bd9f554 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.550056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e7f0-b9af-4b3c-b794-52502bd9f554 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.550355 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e7f0-b9af-4b3c-b794-52502bd9f554 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.564661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74e7f0-b9af-4b3c-b794-52502bd9f554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ab74e7f0-b9af-4b3c-b794-52502bd9f554 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:27.574042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:27.575079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:27.575940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 305/621] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:27.577235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e7f0-b9af-4b3c-b794-52502bd9f554 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:27.646785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50659e-0412-46d2-9c20-211fc879b69c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:27.703921 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1299c6-83b8-45c6-b46a-72c5fcb375ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:27.705029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1299c6-83b8-45c6-b46a-72c5fcb375ae None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.705384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1299c6-83b8-45c6-b46a-72c5fcb375ae None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.705671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1299c6-83b8-45c6-b46a-72c5fcb375ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.725883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e7f0-b9af-4b3c-b794-52502bd9f554 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:27.758798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:27.790185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f24790-7747-4303-9ecc-5b5a61878ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:27.790917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24790-7747-4303-9ecc-5b5a61878ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.791204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24790-7747-4303-9ecc-5b5a61878ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.791485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24790-7747-4303-9ecc-5b5a61878ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.792270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f24790-7747-4303-9ecc-5b5a61878ee9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:27.792581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f24790-7747-4303-9ecc-5b5a61878ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:27.794356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f24790-7747-4303-9ecc-5b5a61878ee9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:27.809632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 306/622] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:27.809632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f24790-7747-4303-9ecc-5b5a61878ee9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:27.815768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1299c6-83b8-45c6-b46a-72c5fcb375ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:27.818290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1299c6-83b8-45c6-b46a-72c5fcb375ae None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:27.828893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:27.928937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f24790-7747-4303-9ecc-5b5a61878ee9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:27.943251 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 317/623] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:27.944512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1299c6-83b8-45c6-b46a-72c5fcb375ae None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:27.974747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27738d68-327e-400b-b1d4-e14af8828c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:27.976081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27738d68-327e-400b-b1d4-e14af8828c43 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:27.976601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27738d68-327e-400b-b1d4-e14af8828c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:27.977095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27738d68-327e-400b-b1d4-e14af8828c43 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:27.978261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27738d68-327e-400b-b1d4-e14af8828c43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:27.978995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27738d68-327e-400b-b1d4-e14af8828c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:27.980730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27738d68-327e-400b-b1d4-e14af8828c43 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:27.990854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 307/624] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:27.992240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27738d68-327e-400b-b1d4-e14af8828c43 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:28.046753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:28.063728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1299c6-83b8-45c6-b46a-72c5fcb375ae None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:28.073547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:28.105620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021d22f-5ee3-4e9a-afe1-2aa91b8fcb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:28.106482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021d22f-5ee3-4e9a-afe1-2aa91b8fcb92 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:28.106776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021d22f-5ee3-4e9a-afe1-2aa91b8fcb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:28.107047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021d22f-5ee3-4e9a-afe1-2aa91b8fcb92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:28.119842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27738d68-327e-400b-b1d4-e14af8828c43 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:28.120070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021d22f-5ee3-4e9a-afe1-2aa91b8fcb92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:28.121422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021d22f-5ee3-4e9a-afe1-2aa91b8fcb92 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0021d22f-5ee3-4e9a-afe1-2aa91b8fcb92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:28.142965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:28.143987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:28.144893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:28.144893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:28.144893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:28.144893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:28.149047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 318/625] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:28.150877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021d22f-5ee3-4e9a-afe1-2aa91b8fcb92 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:28.163509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c8ca-9a98-47d2-b66d-4fde5b829342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:28.164753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c8ca-9a98-47d2-b66d-4fde5b829342 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:28.165811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c8ca-9a98-47d2-b66d-4fde5b829342 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:28.166336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c8ca-9a98-47d2-b66d-4fde5b829342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:28.179641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2c8ca-9a98-47d2-b66d-4fde5b829342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:28.181959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2c8ca-9a98-47d2-b66d-4fde5b829342 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-24f2c8ca-9a98-47d2-b66d-4fde5b829342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:28.202956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:28.203954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:28.204867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:28.204867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:28.204867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:28.204867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:28.212677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 308/626] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:28.213994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c8ca-9a98-47d2-b66d-4fde5b829342 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:28.276752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021d22f-5ee3-4e9a-afe1-2aa91b8fcb92 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:28.327850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:28.329527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca78a7-a28d-4535-a9de-2bcf3c09f032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:28.330523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca78a7-a28d-4535-a9de-2bcf3c09f032 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:28.330834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca78a7-a28d-4535-a9de-2bcf3c09f032 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:28.331271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca78a7-a28d-4535-a9de-2bcf3c09f032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:28.342065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c8ca-9a98-47d2-b66d-4fde5b829342 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:28.345334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca78a7-a28d-4535-a9de-2bcf3c09f032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:28.347345 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca78a7-a28d-4535-a9de-2bcf3c09f032 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-35ca78a7-a28d-4535-a9de-2bcf3c09f032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:28.369475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:28.370701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:28.372544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:28.372544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:28.372544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:28.372544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:28.372544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 319/627] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:28.372544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca78a7-a28d-4535-a9de-2bcf3c09f032 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:28.386021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:28.390514 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5e46e-298f-4b5b-8728-0b5843389e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:28.391550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5e46e-298f-4b5b-8728-0b5843389e5b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:28.392498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5e46e-298f-4b5b-8728-0b5843389e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:28.393002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5e46e-298f-4b5b-8728-0b5843389e5b None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:28.394379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5e46e-298f-4b5b-8728-0b5843389e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:28.394977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5e46e-298f-4b5b-8728-0b5843389e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:28.396614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5e46e-298f-4b5b-8728-0b5843389e5b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:28.411082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 309/628] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:28.412415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5e46e-298f-4b5b-8728-0b5843389e5b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:28.511872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca78a7-a28d-4535-a9de-2bcf3c09f032 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:28.540604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5e46e-298f-4b5b-8728-0b5843389e5b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:28.556492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb2aa1-fdf8-4260-9fc8-b624ae10b40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:28.557328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb2aa1-fdf8-4260-9fc8-b624ae10b40f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:28.557619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb2aa1-fdf8-4260-9fc8-b624ae10b40f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:28.557950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb2aa1-fdf8-4260-9fc8-b624ae10b40f None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:28.558727 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb2aa1-fdf8-4260-9fc8-b624ae10b40f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:28.559040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb2aa1-fdf8-4260-9fc8-b624ae10b40f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:28.560479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb2aa1-fdf8-4260-9fc8-b624ae10b40f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:28.579914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 320/629] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:28.579914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb2aa1-fdf8-4260-9fc8-b624ae10b40f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:28.609685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaacbd-2d04-4a32-b686-3f91422ee2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:28.610783 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaacbd-2d04-4a32-b686-3f91422ee2ce None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:28.611410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaacbd-2d04-4a32-b686-3f91422ee2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:28.611919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaacbd-2d04-4a32-b686-3f91422ee2ce None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:28.612936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcaacbd-2d04-4a32-b686-3f91422ee2ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:28.613477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcaacbd-2d04-4a32-b686-3f91422ee2ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:28.615259 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcaacbd-2d04-4a32-b686-3f91422ee2ce None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:28.631802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 310/630] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:28.633219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaacbd-2d04-4a32-b686-3f91422ee2ce None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:28.653749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:28.672024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:28.715184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb2aa1-fdf8-4260-9fc8-b624ae10b40f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:28.764315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e8e9f-6f8c-4c19-b3c5-c5d4897c0185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:28.765825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e8e9f-6f8c-4c19-b3c5-c5d4897c0185 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:28.766288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e8e9f-6f8c-4c19-b3c5-c5d4897c0185 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:28.766722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e8e9f-6f8c-4c19-b3c5-c5d4897c0185 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:28.777399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e8e9f-6f8c-4c19-b3c5-c5d4897c0185 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:28.779434 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e8e9f-6f8c-4c19-b3c5-c5d4897c0185 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:28.783243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaacbd-2d04-4a32-b686-3f91422ee2ce None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-887e8e9f-6f8c-4c19-b3c5-c5d4897c0185 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:28.798852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:28.800066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:28.801305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:28.810138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 321/631] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:28.812126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e8e9f-6f8c-4c19-b3c5-c5d4897c0185 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:28.831913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ce300-761f-40b1-8dc0-37f8a43837b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:28.833906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce300-761f-40b1-8dc0-37f8a43837b7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:28.835541 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce300-761f-40b1-8dc0-37f8a43837b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:28.835541 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce300-761f-40b1-8dc0-37f8a43837b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:28.844952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ce300-761f-40b1-8dc0-37f8a43837b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:28.846598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ce300-761f-40b1-8dc0-37f8a43837b7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1c4ce300-761f-40b1-8dc0-37f8a43837b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:28.863992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:28.865764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:28.867434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:28.867434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:28.867434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:28.867434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:28.867434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:28.867434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:28.867434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:28.867434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:28.871490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 311/632] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:28.872647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ce300-761f-40b1-8dc0-37f8a43837b7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:28.973261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:28.981857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e8e9f-6f8c-4c19-b3c5-c5d4897c0185 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:29.011575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ce300-761f-40b1-8dc0-37f8a43837b7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:29.022160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:29.032082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2837d3-e2f0-4963-8776-d3a630af6f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:29.033853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2837d3-e2f0-4963-8776-d3a630af6f8d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:29.034806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2837d3-e2f0-4963-8776-d3a630af6f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:29.035680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2837d3-e2f0-4963-8776-d3a630af6f8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:29.049253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2837d3-e2f0-4963-8776-d3a630af6f8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:29.051272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2837d3-e2f0-4963-8776-d3a630af6f8d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:29.058624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97749c-20ef-4691-8b47-0deea3e477af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:29.059311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97749c-20ef-4691-8b47-0deea3e477af None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:29.059596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97749c-20ef-4691-8b47-0deea3e477af None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:29.059875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97749c-20ef-4691-8b47-0deea3e477af None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:29.060742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97749c-20ef-4691-8b47-0deea3e477af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:29.061055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97749c-20ef-4691-8b47-0deea3e477af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:29.064271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97749c-20ef-4691-8b47-0deea3e477af None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-5a2837d3-e2f0-4963-8776-d3a630af6f8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:29.072185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:29.073369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:29.074360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:29.078593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 322/633] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 6 14:08:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:29.080402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2837d3-e2f0-4963-8776-d3a630af6f8d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:29.094681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 312/634] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 6 14:08:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:29.094681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97749c-20ef-4691-8b47-0deea3e477af None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:29.221647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97749c-20ef-4691-8b47-0deea3e477af None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:29.224790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2837d3-e2f0-4963-8776-d3a630af6f8d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:29.274753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fa8be-98ad-4cc8-a496-9398d3ed9b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:29.275860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fa8be-98ad-4cc8-a496-9398d3ed9b72 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:29.276378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fa8be-98ad-4cc8-a496-9398d3ed9b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:29.276876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fa8be-98ad-4cc8-a496-9398d3ed9b72 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:29.277971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fa8be-98ad-4cc8-a496-9398d3ed9b72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:29.278584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fa8be-98ad-4cc8-a496-9398d3ed9b72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:29.280339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fa8be-98ad-4cc8-a496-9398d3ed9b72 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:29.287135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:29.297560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dababad9-8c09-4831-9b1a-d6fa5957ca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:29.298748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dababad9-8c09-4831-9b1a-d6fa5957ca9e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:29.299381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dababad9-8c09-4831-9b1a-d6fa5957ca9e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:29.299804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dababad9-8c09-4831-9b1a-d6fa5957ca9e None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:29.300117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 323/635] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:29.301807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fa8be-98ad-4cc8-a496-9398d3ed9b72 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:29.309162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dababad9-8c09-4831-9b1a-d6fa5957ca9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:29.309680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dababad9-8c09-4831-9b1a-d6fa5957ca9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:29.318215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dababad9-8c09-4831-9b1a-d6fa5957ca9e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:29.357408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 313/636] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 6 14:08:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:29.359362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dababad9-8c09-4831-9b1a-d6fa5957ca9e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:29.378821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:29.446698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fa8be-98ad-4cc8-a496-9398d3ed9b72 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:29.495213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df5a2e-fbc2-4479-883b-9511e52a1eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:29.497090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df5a2e-fbc2-4479-883b-9511e52a1eee None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:29.498030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df5a2e-fbc2-4479-883b-9511e52a1eee None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:29.498600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df5a2e-fbc2-4479-883b-9511e52a1eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:29.513098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df5a2e-fbc2-4479-883b-9511e52a1eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:29.515810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df5a2e-fbc2-4479-883b-9511e52a1eee None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-83df5a2e-fbc2-4479-883b-9511e52a1eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:29.542499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:29.543502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:29.544412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:29.544412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:29.544412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:29.544412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:29.552274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 324/637] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:29.553666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df5a2e-fbc2-4479-883b-9511e52a1eee None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:29.599259 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dababad9-8c09-4831-9b1a-d6fa5957ca9e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:29.684520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9a487-250a-4251-8722-36a989f19b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:29.690629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9a487-250a-4251-8722-36a989f19b4e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:29.691379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9a487-250a-4251-8722-36a989f19b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:29.692055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9a487-250a-4251-8722-36a989f19b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:29.693154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df5a2e-fbc2-4479-883b-9511e52a1eee None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:29.694325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9a487-250a-4251-8722-36a989f19b4e None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=f41a357957bc47739b7bd6f21d159d74) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:29.694978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9a487-250a-4251-8722-36a989f19b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:29.698549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:29.701292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9a487-250a-4251-8722-36a989f19b4e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:29.721185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 314/638] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:29 2022] GET /identity/v3/role_assignments?user.id=f41a357957bc47739b7bd6f21d159d74&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 608 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:29.723302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9a487-250a-4251-8722-36a989f19b4e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:29.737849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4495ba-c159-48e6-a795-a5dea0e35856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:29.738614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4495ba-c159-48e6-a795-a5dea0e35856 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:29.739014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4495ba-c159-48e6-a795-a5dea0e35856 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:29.739444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4495ba-c159-48e6-a795-a5dea0e35856 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:29.833746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4495ba-c159-48e6-a795-a5dea0e35856 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:29.834577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4495ba-c159-48e6-a795-a5dea0e35856 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:29.864848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:29.947593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9a487-250a-4251-8722-36a989f19b4e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:29.998633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 325/639] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:30.002077 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216846a-e44b-486e-99af-a66cd919c894 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.002873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216846a-e44b-486e-99af-a66cd919c894 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.003551 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216846a-e44b-486e-99af-a66cd919c894 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.006624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 326/640] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:30.008232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fbc67-bcfc-4c4d-ba76-1b5efd1c5c5b None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.008692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fbc67-bcfc-4c4d-ba76-1b5efd1c5c5b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.009573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fbc67-bcfc-4c4d-ba76-1b5efd1c5c5b None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.011401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 327/641] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:30.016692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fbc67-bcfc-4c4d-ba76-1b5efd1c5c5b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:30.017529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:30.021847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc98fe-0fbd-4685-a0c0-b2c6da25a17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:30.022937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc98fe-0fbd-4685-a0c0-b2c6da25a17e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.023421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc98fe-0fbd-4685-a0c0-b2c6da25a17e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.023878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc98fe-0fbd-4685-a0c0-b2c6da25a17e None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.025609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc98fe-0fbd-4685-a0c0-b2c6da25a17e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:30.026004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc98fe-0fbd-4685-a0c0-b2c6da25a17e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:30.029414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc98fe-0fbd-4685-a0c0-b2c6da25a17e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:30.054676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 315/642] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:30.057594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0cfbf-7dc2-470e-943b-6ddca437cc2c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.058106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0cfbf-7dc2-470e-943b-6ddca437cc2c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.058583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0cfbf-7dc2-470e-943b-6ddca437cc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.086148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e5d0cfbf-7dc2-470e-943b-6ddca437cc2c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:30.088527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e5d0cfbf-7dc2-470e-943b-6ddca437cc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:30.184922 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fbc67-bcfc-4c4d-ba76-1b5efd1c5c5b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:30.200808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 316/643] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:30.203477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f9503-53c2-4b3a-beda-9c4cf0d7cb65 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.204083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f9503-53c2-4b3a-beda-9c4cf0d7cb65 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.204731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f9503-53c2-4b3a-beda-9c4cf0d7cb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.234974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ce3f9503-53c2-4b3a-beda-9c4cf0d7cb65 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:30.236983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ce3f9503-53c2-4b3a-beda-9c4cf0d7cb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:30.247568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48594c-0dfc-4d26-bb05-0315c19cfbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:30.249336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48594c-0dfc-4d26-bb05-0315c19cfbbe None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.249824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48594c-0dfc-4d26-bb05-0315c19cfbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.250252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48594c-0dfc-4d26-bb05-0315c19cfbbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.269500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48594c-0dfc-4d26-bb05-0315c19cfbbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:30.271669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48594c-0dfc-4d26-bb05-0315c19cfbbe None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-de48594c-0dfc-4d26-bb05-0315c19cfbbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:30.303206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:30.314587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:30.316313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:30.316313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:30.316313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:30.316313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:30.316313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 328/644] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:30.316313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48594c-0dfc-4d26-bb05-0315c19cfbbe None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:30.317076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:30.351110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 317/645] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:30.352199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f9503-53c2-4b3a-beda-9c4cf0d7cb65 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:30.389796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:30.455348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48594c-0dfc-4d26-bb05-0315c19cfbbe None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:30.480635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f9503-53c2-4b3a-beda-9c4cf0d7cb65 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:30.505135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee84d8-7583-4aa8-8ac7-7293d99d9ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:30.506201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee84d8-7583-4aa8-8ac7-7293d99d9ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.506599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee84d8-7583-4aa8-8ac7-7293d99d9ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.506991 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee84d8-7583-4aa8-8ac7-7293d99d9ed1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.516935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee84d8-7583-4aa8-8ac7-7293d99d9ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:30.518698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee84d8-7583-4aa8-8ac7-7293d99d9ed1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:30.529205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a41418-fa51-4476-be79-0dbd09c13777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:30.530056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a41418-fa51-4476-be79-0dbd09c13777 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.530349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a41418-fa51-4476-be79-0dbd09c13777 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.530606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a41418-fa51-4476-be79-0dbd09c13777 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f5ee84d8-7583-4aa8-8ac7-7293d99d9ed1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:30.534713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:30.535745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 329/646] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:30.537484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee84d8-7583-4aa8-8ac7-7293d99d9ed1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:30.556564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a41418-fa51-4476-be79-0dbd09c13777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-58a41418-fa51-4476-be79-0dbd09c13777 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:30.566278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:30.567301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:30.568157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:30.574798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 318/647] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:30.575969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a41418-fa51-4476-be79-0dbd09c13777 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:30.635423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:30.697888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee84d8-7583-4aa8-8ac7-7293d99d9ed1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:30.706509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a41418-fa51-4476-be79-0dbd09c13777 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:30.722435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:30.745022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb3dae-4c08-492a-9b53-8fd4b62d70ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:30.746133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb3dae-4c08-492a-9b53-8fd4b62d70ff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.746407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb3dae-4c08-492a-9b53-8fd4b62d70ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.746803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb3dae-4c08-492a-9b53-8fd4b62d70ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.753620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bf53f-89a5-47ee-98cb-b8958f9ef8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:30.754590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bf53f-89a5-47ee-98cb-b8958f9ef8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.754876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bf53f-89a5-47ee-98cb-b8958f9ef8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.755559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bf53f-89a5-47ee-98cb-b8958f9ef8f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.756296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bf53f-89a5-47ee-98cb-b8958f9ef8f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:30.756593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bf53f-89a5-47ee-98cb-b8958f9ef8f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:30.758232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb3dae-4c08-492a-9b53-8fd4b62d70ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:30.758404 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bf53f-89a5-47ee-98cb-b8958f9ef8f8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-3eeb3dae-4c08-492a-9b53-8fd4b62d70ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:30.766210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:30.767240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:30.768096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:30.771557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 319/648] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:30.773993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb3dae-4c08-492a-9b53-8fd4b62d70ff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:30.775682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 330/649] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:30.776649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bf53f-89a5-47ee-98cb-b8958f9ef8f8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:30.885945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:30.894557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb3dae-4c08-492a-9b53-8fd4b62d70ff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:30.932889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d088505-1dc1-4365-a1af-929dd95c4a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:30.933583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d088505-1dc1-4365-a1af-929dd95c4a27 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.933892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d088505-1dc1-4365-a1af-929dd95c4a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.934162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d088505-1dc1-4365-a1af-929dd95c4a27 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.934971 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d088505-1dc1-4365-a1af-929dd95c4a27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:30.935274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d088505-1dc1-4365-a1af-929dd95c4a27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:30.936755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d088505-1dc1-4365-a1af-929dd95c4a27 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:30.945653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bf53f-89a5-47ee-98cb-b8958f9ef8f8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:30.959802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 320/650] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:30.959802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d088505-1dc1-4365-a1af-929dd95c4a27 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:30.990497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496116dd-dab5-44f4-b8c8-eeeb683f34ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:30.991091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496116dd-dab5-44f4-b8c8-eeeb683f34ea None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:30.991365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496116dd-dab5-44f4-b8c8-eeeb683f34ea None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:30.991630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496116dd-dab5-44f4-b8c8-eeeb683f34ea None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:30.992389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496116dd-dab5-44f4-b8c8-eeeb683f34ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:30.992694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496116dd-dab5-44f4-b8c8-eeeb683f34ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:30.994706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496116dd-dab5-44f4-b8c8-eeeb683f34ea None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:31.003941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 331/651] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:31.004839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496116dd-dab5-44f4-b8c8-eeeb683f34ea None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:31.050981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:31.086999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d088505-1dc1-4365-a1af-929dd95c4a27 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:31.125890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54472c68-148d-453f-b9a9-fd817d171712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:31.127283 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54472c68-148d-453f-b9a9-fd817d171712 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:31.127608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54472c68-148d-453f-b9a9-fd817d171712 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:31.128068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54472c68-148d-453f-b9a9-fd817d171712 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:31.129486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54472c68-148d-453f-b9a9-fd817d171712 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:31.130140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54472c68-148d-453f-b9a9-fd817d171712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:31.130620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496116dd-dab5-44f4-b8c8-eeeb683f34ea None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:31.132718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54472c68-148d-453f-b9a9-fd817d171712 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:31.148216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 321/652] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:31.149687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54472c68-148d-453f-b9a9-fd817d171712 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:31.172689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9eef6f-60c9-48e9-8324-4d29c5ca42c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:31.173779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9eef6f-60c9-48e9-8324-4d29c5ca42c5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:31.174374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9eef6f-60c9-48e9-8324-4d29c5ca42c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:31.174798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9eef6f-60c9-48e9-8324-4d29c5ca42c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:31.181585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:31.184104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9eef6f-60c9-48e9-8324-4d29c5ca42c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:31.185989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9eef6f-60c9-48e9-8324-4d29c5ca42c5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2e9eef6f-60c9-48e9-8324-4d29c5ca42c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:31.203329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:31.204493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:31.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:31.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:31.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:31.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:31.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:31.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:31.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:31.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:31.207692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 332/653] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:31.208652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9eef6f-60c9-48e9-8324-4d29c5ca42c5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:31.322134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:31.329548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9eef6f-60c9-48e9-8324-4d29c5ca42c5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:31.330578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54472c68-148d-453f-b9a9-fd817d171712 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:31.371653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939b112-2661-456e-88ef-8f5e31cd221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:31.372455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b112-2661-456e-88ef-8f5e31cd221c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:31.372801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b112-2661-456e-88ef-8f5e31cd221c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:31.372990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b112-2661-456e-88ef-8f5e31cd221c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:31.376168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b28ae-0406-41e0-9dcd-1661b1c3b033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:31.377064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b28ae-0406-41e0-9dcd-1661b1c3b033 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:31.377348 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b28ae-0406-41e0-9dcd-1661b1c3b033 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:31.377631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b28ae-0406-41e0-9dcd-1661b1c3b033 None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/3571f3d3a0de4e9abe55ed84426143a0/roles/4107928942534d3984671dc5232d061d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:31.385482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939b112-2661-456e-88ef-8f5e31cd221c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:31.387106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939b112-2661-456e-88ef-8f5e31cd221c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:31.394693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b28ae-0406-41e0-9dcd-1661b1c3b033 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=3571f3d3a0de4e9abe55ed84426143a0, role_id=4107928942534d3984671dc5232d061d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:31.396189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b28ae-0406-41e0-9dcd-1661b1c3b033 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a939b112-2661-456e-88ef-8f5e31cd221c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:31.408161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:31.410384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:31.412239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:31.412239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:31.412239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:31.412239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:31.412239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 333/654] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:31.412239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939b112-2661-456e-88ef-8f5e31cd221c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:31.418256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 322/655] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:08:31 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/3571f3d3a0de4e9abe55ed84426143a0/roles/4107928942534d3984671dc5232d061d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:08:31.418736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b28ae-0406-41e0-9dcd-1661b1c3b033 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:31.550003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:31.617449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939b112-2661-456e-88ef-8f5e31cd221c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:31.627540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b28ae-0406-41e0-9dcd-1661b1c3b033 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:31.690164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c104c-2d69-49f7-b355-71707e8dd90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:31.691741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c104c-2d69-49f7-b355-71707e8dd90f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:31.692270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611a215-50fa-4b07-8adf-6ddf0c586d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:31.692457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c104c-2d69-49f7-b355-71707e8dd90f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:31.692997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c104c-2d69-49f7-b355-71707e8dd90f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:31.693691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611a215-50fa-4b07-8adf-6ddf0c586d7e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:31.696011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611a215-50fa-4b07-8adf-6ddf0c586d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:31.696011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611a215-50fa-4b07-8adf-6ddf0c586d7e None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:31.698025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611a215-50fa-4b07-8adf-6ddf0c586d7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:31.698598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611a215-50fa-4b07-8adf-6ddf0c586d7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:31.701151 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611a215-50fa-4b07-8adf-6ddf0c586d7e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:31.715712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c104c-2d69-49f7-b355-71707e8dd90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:31.719287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c104c-2d69-49f7-b355-71707e8dd90f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:31.725180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 323/656] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:31.726491 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611a215-50fa-4b07-8adf-6ddf0c586d7e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:31.738978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-378c104c-2d69-49f7-b355-71707e8dd90f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:31.751342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:31.753080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:31.754900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:31.754900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:31.754900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:31.754900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:31.754900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 334/657] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:31.754900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c104c-2d69-49f7-b355-71707e8dd90f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:31.902164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c104c-2d69-49f7-b355-71707e8dd90f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:31.938555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611a215-50fa-4b07-8adf-6ddf0c586d7e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:31.955460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b10e5-83df-4bb4-8b41-4714ea9a3f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:31.956624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b10e5-83df-4bb4-8b41-4714ea9a3f47 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:31.957137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b10e5-83df-4bb4-8b41-4714ea9a3f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:31.957743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b10e5-83df-4bb4-8b41-4714ea9a3f47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:31.959475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b10e5-83df-4bb4-8b41-4714ea9a3f47 None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=0b82a920df4042dd97314350aed4d7a5) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:31.960048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b10e5-83df-4bb4-8b41-4714ea9a3f47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:31.964079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b10e5-83df-4bb4-8b41-4714ea9a3f47 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:31.974522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:31.981872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 335/658] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:31 2022] GET /identity/v3/role_assignments?user.id=0b82a920df4042dd97314350aed4d7a5&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:31.983497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b10e5-83df-4bb4-8b41-4714ea9a3f47 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:31.988964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a13d5-30d8-4870-ada1-0a4b1abf67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:31.990272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a13d5-30d8-4870-ada1-0a4b1abf67b8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:31.990791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a13d5-30d8-4870-ada1-0a4b1abf67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:31.991267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a13d5-30d8-4870-ada1-0a4b1abf67b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:31.992235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a13d5-30d8-4870-ada1-0a4b1abf67b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:31.992765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a13d5-30d8-4870-ada1-0a4b1abf67b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:31.994903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a13d5-30d8-4870-ada1-0a4b1abf67b8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:32.011940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 324/659] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:32.020255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a13d5-30d8-4870-ada1-0a4b1abf67b8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:32.163732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:32.258972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b10e5-83df-4bb4-8b41-4714ea9a3f47 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:32.269344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a13d5-30d8-4870-ada1-0a4b1abf67b8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:32.346605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b537f-a396-4be8-9707-c97d5f8c72bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:32.348170 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b537f-a396-4be8-9707-c97d5f8c72bb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:32.348823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b537f-a396-4be8-9707-c97d5f8c72bb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:32.349464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b537f-a396-4be8-9707-c97d5f8c72bb None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:32.350499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13d0f3-2c61-4a33-90ac-e419616cae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:32.351306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13d0f3-2c61-4a33-90ac-e419616cae02 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:32.351586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13d0f3-2c61-4a33-90ac-e419616cae02 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:32.351864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13d0f3-2c61-4a33-90ac-e419616cae02 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:32.352266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b537f-a396-4be8-9707-c97d5f8c72bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:32.353885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b537f-a396-4be8-9707-c97d5f8c72bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:32.356864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b537f-a396-4be8-9707-c97d5f8c72bb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:32.362418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13d0f3-2c61-4a33-90ac-e419616cae02 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:32.363603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13d0f3-2c61-4a33-90ac-e419616cae02 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:32.381250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 336/660] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1d13d0f3-2c61-4a33-90ac-e419616cae02 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:32.385868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:32.387758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:32.389666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 325/661] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:32.394232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b537f-a396-4be8-9707-c97d5f8c72bb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:32.394704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13d0f3-2c61-4a33-90ac-e419616cae02 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:32.476222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:32.603303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13d0f3-2c61-4a33-90ac-e419616cae02 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:32.617356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b537f-a396-4be8-9707-c97d5f8c72bb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:32.650104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:32.656083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17331110-f25c-4ad3-9a6a-c7edde35a1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:32.656767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17331110-f25c-4ad3-9a6a-c7edde35a1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:32.657048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17331110-f25c-4ad3-9a6a-c7edde35a1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:32.657323 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17331110-f25c-4ad3-9a6a-c7edde35a1a0 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:32.662372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17331110-f25c-4ad3-9a6a-c7edde35a1a0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:32.662704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17331110-f25c-4ad3-9a6a-c7edde35a1a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:32.663892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d5a4f-3001-4143-89dc-f7cce297cb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:32.664213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17331110-f25c-4ad3-9a6a-c7edde35a1a0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:32.664715 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d5a4f-3001-4143-89dc-f7cce297cb16 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:32.665606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d5a4f-3001-4143-89dc-f7cce297cb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:32.665924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d5a4f-3001-4143-89dc-f7cce297cb16 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:32.676467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d5a4f-3001-4143-89dc-f7cce297cb16 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:32.678014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d5a4f-3001-4143-89dc-f7cce297cb16 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0b3d5a4f-3001-4143-89dc-f7cce297cb16 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:32.695520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:32.697426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 14:08:32.700546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:32.702724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 337/662] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:32.705741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d5a4f-3001-4143-89dc-f7cce297cb16 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:32.720286 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 326/663] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 6 14:08:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:32.729228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17331110-f25c-4ad3-9a6a-c7edde35a1a0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:32.841433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d5a4f-3001-4143-89dc-f7cce297cb16 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:32.885686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bb0f7-8741-4838-8212-982b3b419a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:32.886436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bb0f7-8741-4838-8212-982b3b419a37 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:32.886737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bb0f7-8741-4838-8212-982b3b419a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:32.887013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bb0f7-8741-4838-8212-982b3b419a37 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:32.887894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bb0f7-8741-4838-8212-982b3b419a37 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:32.888205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bb0f7-8741-4838-8212-982b3b419a37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:32.889793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bb0f7-8741-4838-8212-982b3b419a37 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:32.918105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 338/664] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Jun 6 14:08:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:32.920494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bb0f7-8741-4838-8212-982b3b419a37 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:32.982523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:33.044855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bb0f7-8741-4838-8212-982b3b419a37 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:33.086483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e549-4ade-49b7-ad55-262b7beea01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:33.088570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e549-4ade-49b7-ad55-262b7beea01e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:33.088882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e549-4ade-49b7-ad55-262b7beea01e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:33.089139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e549-4ade-49b7-ad55-262b7beea01e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:33.139819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17331110-f25c-4ad3-9a6a-c7edde35a1a0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:33.148587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:33.178294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18e549-4ade-49b7-ad55-262b7beea01e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:33.179918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18e549-4ade-49b7-ad55-262b7beea01e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:33.232245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:33.252259 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a06d4f-5cc3-4cf4-8556-b5242cf07fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:33.253302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06d4f-5cc3-4cf4-8556-b5242cf07fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:33.253840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06d4f-5cc3-4cf4-8556-b5242cf07fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:33.254387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06d4f-5cc3-4cf4-8556-b5242cf07fc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:33.311928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 339/665] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 14:08:33.313366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e549-4ade-49b7-ad55-262b7beea01e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:33.376937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a06d4f-5cc3-4cf4-8556-b5242cf07fc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:33.378916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a06d4f-5cc3-4cf4-8556-b5242cf07fc8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:33.437860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e549-4ade-49b7-ad55-262b7beea01e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:33.480061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74653917-dcf8-4009-9db4-deea461d09df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:33.480901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74653917-dcf8-4009-9db4-deea461d09df None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:33.481190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74653917-dcf8-4009-9db4-deea461d09df None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:33.481468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74653917-dcf8-4009-9db4-deea461d09df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:33.493760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74653917-dcf8-4009-9db4-deea461d09df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:33.495306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74653917-dcf8-4009-9db4-deea461d09df None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:33.497402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:33.508108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 327/666] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 779 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:33.510580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a55240-2bf4-47a1-a923-4b7c5927f91e None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:33.511181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a55240-2bf4-47a1-a923-4b7c5927f91e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:33.511685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a55240-2bf4-47a1-a923-4b7c5927f91e None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:33.517739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 328/667] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:33.519067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a55240-2bf4-47a1-a923-4b7c5927f91e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-74653917-dcf8-4009-9db4-deea461d09df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:33.520752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:33.521674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:33.522932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:33.522932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:33.522932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:33.522932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:33.526551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:33.526840 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 340/668] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:33.528681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cdd17-9695-4f6e-b050-207882f34b71 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:33.528844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cdd17-9695-4f6e-b050-207882f34b71 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:33.529029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cdd17-9695-4f6e-b050-207882f34b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:33.551195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c82cdd17-9695-4f6e-b050-207882f34b71 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:33.552329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c82cdd17-9695-4f6e-b050-207882f34b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:33.637895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 341/669] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:33.637895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cdd17-9695-4f6e-b050-207882f34b71 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:33.671565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a55240-2bf4-47a1-a923-4b7c5927f91e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:33.720699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932613e6-6f4a-4b73-9f76-d97e4c46db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:33.721979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932613e6-6f4a-4b73-9f76-d97e4c46db47 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:33.722481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932613e6-6f4a-4b73-9f76-d97e4c46db47 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:33.722959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932613e6-6f4a-4b73-9f76-d97e4c46db47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:33.735402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932613e6-6f4a-4b73-9f76-d97e4c46db47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:33.737287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932613e6-6f4a-4b73-9f76-d97e4c46db47 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:33.752231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cdd17-9695-4f6e-b050-207882f34b71 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-932613e6-6f4a-4b73-9f76-d97e4c46db47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:33.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:33.760517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:33.761458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:33.761458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:33.761458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:33.761458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:33.764017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 329/670] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:33.765241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932613e6-6f4a-4b73-9f76-d97e4c46db47 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:33.792490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b749ea4-22a9-41c5-a142-caed4ae56476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:33.793178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b749ea4-22a9-41c5-a142-caed4ae56476 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:33.793485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b749ea4-22a9-41c5-a142-caed4ae56476 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:33.793813 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b749ea4-22a9-41c5-a142-caed4ae56476 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:33.795235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b749ea4-22a9-41c5-a142-caed4ae56476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:33.795235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b749ea4-22a9-41c5-a142-caed4ae56476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:33.796670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b749ea4-22a9-41c5-a142-caed4ae56476 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:33.810053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:33.814349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 342/671] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:33.814871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b749ea4-22a9-41c5-a142-caed4ae56476 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:33.834433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:33.889093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932613e6-6f4a-4b73-9f76-d97e4c46db47 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:33.928914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8bfad-cd70-4a48-9e89-ed34749951a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:33.930192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8bfad-cd70-4a48-9e89-ed34749951a1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:33.930331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b749ea4-22a9-41c5-a142-caed4ae56476 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:33.931603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8bfad-cd70-4a48-9e89-ed34749951a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:33.932467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8bfad-cd70-4a48-9e89-ed34749951a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:33.944996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8bfad-cd70-4a48-9e89-ed34749951a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:33.947016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8bfad-cd70-4a48-9e89-ed34749951a1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e7d8bfad-cd70-4a48-9e89-ed34749951a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:33.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:33.968677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:33.969677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:33.969677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:33.969677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:33.969677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:33.972091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3478e3-1cdd-4255-8665-f23abc372ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:33.972784 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3478e3-1cdd-4255-8665-f23abc372ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:33.973073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3478e3-1cdd-4255-8665-f23abc372ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:33.973455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3478e3-1cdd-4255-8665-f23abc372ab3 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:33.978425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3478e3-1cdd-4255-8665-f23abc372ab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:33.978425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3478e3-1cdd-4255-8665-f23abc372ab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:33.978425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3478e3-1cdd-4255-8665-f23abc372ab3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:33.979234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 330/672] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:33.980497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8bfad-cd70-4a48-9e89-ed34749951a1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:33.997456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 343/673] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:33.998506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3478e3-1cdd-4255-8665-f23abc372ab3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:34.078680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:34.099352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8bfad-cd70-4a48-9e89-ed34749951a1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:34.111016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:34.119400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3478e3-1cdd-4255-8665-f23abc372ab3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:34.144071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb22f6-2246-4972-ab80-99e3b9f6082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:34.145526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb22f6-2246-4972-ab80-99e3b9f6082f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:34.146104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb22f6-2246-4972-ab80-99e3b9f6082f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:34.146637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb22f6-2246-4972-ab80-99e3b9f6082f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:34.156278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb22f6-2246-4972-ab80-99e3b9f6082f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:34.158280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb22f6-2246-4972-ab80-99e3b9f6082f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:34.165470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc251839-fc73-47a7-a661-b4ec89b11d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:34.166197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc251839-fc73-47a7-a661-b4ec89b11d82 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:34.166499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc251839-fc73-47a7-a661-b4ec89b11d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:34.166754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc251839-fc73-47a7-a661-b4ec89b11d82 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:34.167624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc251839-fc73-47a7-a661-b4ec89b11d82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:34.167825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc251839-fc73-47a7-a661-b4ec89b11d82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:34.169211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc251839-fc73-47a7-a661-b4ec89b11d82 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-87bb22f6-2246-4972-ab80-99e3b9f6082f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:34.176965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:34.178023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:34.178973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:34.178973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:34.178973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:34.178973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:34.178973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:34.178973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:34.178973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:34.178973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:34.182978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 331/674] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:34.184235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb22f6-2246-4972-ab80-99e3b9f6082f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:34.186195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 344/675] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:34.194256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc251839-fc73-47a7-a661-b4ec89b11d82 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:34.331957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb22f6-2246-4972-ab80-99e3b9f6082f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:34.333993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc251839-fc73-47a7-a661-b4ec89b11d82 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:34.336860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:34.377127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5c7fb-d356-458e-b1d2-05a742d6f3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:34.377343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce8ae0-0df9-438e-aa2f-695ffac288d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:34.377912 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce8ae0-0df9-438e-aa2f-695ffac288d5 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:34.378204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce8ae0-0df9-438e-aa2f-695ffac288d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:34.378487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce8ae0-0df9-438e-aa2f-695ffac288d5 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:34.379230 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce8ae0-0df9-438e-aa2f-695ffac288d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:34.379506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5c7fb-d356-458e-b1d2-05a742d6f3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:34.380033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5c7fb-d356-458e-b1d2-05a742d6f3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:34.380513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5c7fb-d356-458e-b1d2-05a742d6f3a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:34.380735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce8ae0-0df9-438e-aa2f-695ffac288d5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:34.390439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5c7fb-d356-458e-b1d2-05a742d6f3a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:34.392707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5c7fb-d356-458e-b1d2-05a742d6f3a2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2fb5c7fb-d356-458e-b1d2-05a742d6f3a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:34.410216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:34.411225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:34.412172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:34.412172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:34.412172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:34.412172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:34.412172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:34.412172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:34.412172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:34.412172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:34.413817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 345/676] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:08:34 2022] POST /identity/v3/users => generated 254 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:08:34.415529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce8ae0-0df9-438e-aa2f-695ffac288d5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:34.416541 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 332/677] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:34.432934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5c7fb-d356-458e-b1d2-05a742d6f3a2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:34.489455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:34.557884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce8ae0-0df9-438e-aa2f-695ffac288d5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:34.602738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd59b5-6036-403c-9e3b-b94522f6d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:34.604053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd59b5-6036-403c-9e3b-b94522f6d659 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:34.604724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd59b5-6036-403c-9e3b-b94522f6d659 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:34.605356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd59b5-6036-403c-9e3b-b94522f6d659 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:34.606668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd59b5-6036-403c-9e3b-b94522f6d659 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:34.607171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd59b5-6036-403c-9e3b-b94522f6d659 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:34.609792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd59b5-6036-403c-9e3b-b94522f6d659 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:34.632106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 346/678] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:34.640554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f3216-86e1-448c-a213-f708fd6d5470 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:34.640953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f3216-86e1-448c-a213-f708fd6d5470 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:34.641424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f3216-86e1-448c-a213-f708fd6d5470 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:34.648492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5c7fb-d356-458e-b1d2-05a742d6f3a2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:34.648726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 347/679] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:34 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:34.649685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f3216-86e1-448c-a213-f708fd6d5470 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:34.691203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:34.722338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e063a-6756-4f9f-a4ba-0266f77aa8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:34.723403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e063a-6756-4f9f-a4ba-0266f77aa8cb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:34.723855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e063a-6756-4f9f-a4ba-0266f77aa8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:34.724291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e063a-6756-4f9f-a4ba-0266f77aa8cb None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:34.725686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e063a-6756-4f9f-a4ba-0266f77aa8cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:34.726223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e063a-6756-4f9f-a4ba-0266f77aa8cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:34.729436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e063a-6756-4f9f-a4ba-0266f77aa8cb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:34.753680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 333/680] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:34.756165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aedcfb-9c99-4ca8-bc1f-bdc5047fd485 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:34.756662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aedcfb-9c99-4ca8-bc1f-bdc5047fd485 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:34.757147 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aedcfb-9c99-4ca8-bc1f-bdc5047fd485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:34.787026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-09aedcfb-9c99-4ca8-bc1f-bdc5047fd485 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:34.788819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-09aedcfb-9c99-4ca8-bc1f-bdc5047fd485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:34.856164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f3216-86e1-448c-a213-f708fd6d5470 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:34.900267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 334/681] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:34.905495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aedcfb-9c99-4ca8-bc1f-bdc5047fd485 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:34.926684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdd5a0-e903-4bb1-b071-c0a1654f07c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:34.928324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdd5a0-e903-4bb1-b071-c0a1654f07c5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:34.929031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdd5a0-e903-4bb1-b071-c0a1654f07c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:34.929684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdd5a0-e903-4bb1-b071-c0a1654f07c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:34.931427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdd5a0-e903-4bb1-b071-c0a1654f07c5 None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=59cd9d922204496d8ea185716b832481) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:34.932330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdd5a0-e903-4bb1-b071-c0a1654f07c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:34.935769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdd5a0-e903-4bb1-b071-c0a1654f07c5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:34.941167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:34.954162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 348/682] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:34 2022] GET /identity/v3/role_assignments?user.id=59cd9d922204496d8ea185716b832481&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:34.971587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdd5a0-e903-4bb1-b071-c0a1654f07c5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:35.150127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aedcfb-9c99-4ca8-bc1f-bdc5047fd485 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:35.160626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:35.195776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdd5a0-e903-4bb1-b071-c0a1654f07c5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:35.234310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cd84b-bce0-4f92-b627-dbadd1b163b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:35.236428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cd84b-bce0-4f92-b627-dbadd1b163b5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:35.237223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cd84b-bce0-4f92-b627-dbadd1b163b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:35.237961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cd84b-bce0-4f92-b627-dbadd1b163b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:35.239835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cd84b-bce0-4f92-b627-dbadd1b163b5 None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=720c995318a24e7b859fa82febd8178b) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:35.241620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cd84b-bce0-4f92-b627-dbadd1b163b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:35.245687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cd84b-bce0-4f92-b627-dbadd1b163b5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:35.261102 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 335/683] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:34 2022] GET /identity/v3/role_assignments?user.id=720c995318a24e7b859fa82febd8178b&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 608 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:35.269763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27c0e7-8126-4712-b4c2-4832c2aaeeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:35.271603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27c0e7-8126-4712-b4c2-4832c2aaeeca None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:35.272368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27c0e7-8126-4712-b4c2-4832c2aaeeca None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:35.273099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27c0e7-8126-4712-b4c2-4832c2aaeeca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:35.295952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27c0e7-8126-4712-b4c2-4832c2aaeeca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1d27c0e7-8126-4712-b4c2-4832c2aaeeca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:35.314400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:35.316366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:35.318047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:35.322532 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 349/684] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:35.332043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cd84b-bce0-4f92-b627-dbadd1b163b5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:35.534764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:35.583057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cd84b-bce0-4f92-b627-dbadd1b163b5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:35.641918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f46f8-5726-41be-9a26-177262f60874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:35.642594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f46f8-5726-41be-9a26-177262f60874 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:35.642878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f46f8-5726-41be-9a26-177262f60874 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:35.643287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f46f8-5726-41be-9a26-177262f60874 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:35.644060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f46f8-5726-41be-9a26-177262f60874 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:35.644369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f46f8-5726-41be-9a26-177262f60874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:35.645787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f46f8-5726-41be-9a26-177262f60874 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:35.655835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 336/685] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:35.667669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27c0e7-8126-4712-b4c2-4832c2aaeeca None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:35.871151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27c0e7-8126-4712-b4c2-4832c2aaeeca None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:35.938801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd7482-1269-4f22-a846-3be71bb251e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:35.940150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd7482-1269-4f22-a846-3be71bb251e7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:35.940662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd7482-1269-4f22-a846-3be71bb251e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:35.941159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd7482-1269-4f22-a846-3be71bb251e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:35.964240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd7482-1269-4f22-a846-3be71bb251e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:35.967319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd7482-1269-4f22-a846-3be71bb251e7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:35.972630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0fbd7482-1269-4f22-a846-3be71bb251e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:36.010592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:36.011583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:36.012509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:36.012509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:36.012509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:36.012509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:36.014782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 350/686] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:36.022456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f46f8-5726-41be-9a26-177262f60874 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:36.022960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7d4be-5b8d-4d8a-82e4-570f6b55cbad None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:36.023254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7d4be-5b8d-4d8a-82e4-570f6b55cbad None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:36.023537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7d4be-5b8d-4d8a-82e4-570f6b55cbad None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:36.025466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 351/687] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:08:36.031547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91865654-c9bb-481f-8a5a-0871d7f52cce None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:36.031833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91865654-c9bb-481f-8a5a-0871d7f52cce None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:36.032114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91865654-c9bb-481f-8a5a-0871d7f52cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:36.061834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-91865654-c9bb-481f-8a5a-0871d7f52cce None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:36.063059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-91865654-c9bb-481f-8a5a-0871d7f52cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:36.201605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 352/688] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:08:36.233620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f46f8-5726-41be-9a26-177262f60874 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:36.249748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91865654-c9bb-481f-8a5a-0871d7f52cce None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:36.299835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f1a8-eedf-409d-8b2c-20c0db926ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:36.301464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f1a8-eedf-409d-8b2c-20c0db926ddc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:36.302014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f1a8-eedf-409d-8b2c-20c0db926ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:36.302503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f1a8-eedf-409d-8b2c-20c0db926ddc None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:36.303880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f1a8-eedf-409d-8b2c-20c0db926ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:36.304428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f1a8-eedf-409d-8b2c-20c0db926ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:36.308033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f1a8-eedf-409d-8b2c-20c0db926ddc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:36.337489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 337/689] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:36.349878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f1a8-eedf-409d-8b2c-20c0db926ddc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:36.364039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:36.526438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91865654-c9bb-481f-8a5a-0871d7f52cce None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:36.532912 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f1a8-eedf-409d-8b2c-20c0db926ddc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:36.579553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:36.584161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde5acc-eecd-4afb-a811-35f36b785698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:36.585029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde5acc-eecd-4afb-a811-35f36b785698 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:36.585312 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde5acc-eecd-4afb-a811-35f36b785698 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:36.585801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde5acc-eecd-4afb-a811-35f36b785698 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:36.596141 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057e31d-4179-43c0-80da-917c71cb6c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:36.597489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e31d-4179-43c0-80da-917c71cb6c4f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:36.598039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e31d-4179-43c0-80da-917c71cb6c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:36.598594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e31d-4179-43c0-80da-917c71cb6c4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:36.610855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde5acc-eecd-4afb-a811-35f36b785698 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:36.612387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde5acc-eecd-4afb-a811-35f36b785698 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:36.622414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057e31d-4179-43c0-80da-917c71cb6c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-3057e31d-4179-43c0-80da-917c71cb6c4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:36.639686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:36.640725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:36.641571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:36.643544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 353/690] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8bde5acc-eecd-4afb-a811-35f36b785698 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:36.648073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:36.650512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:36.652886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 338/691] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:36.657409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057e31d-4179-43c0-80da-917c71cb6c4f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:36.660395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde5acc-eecd-4afb-a811-35f36b785698 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:36.793016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:36.871184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057e31d-4179-43c0-80da-917c71cb6c4f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:36.896444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde5acc-eecd-4afb-a811-35f36b785698 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:36.947464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3319caf-0e62-4f53-b347-158ade163929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:36.950290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3319caf-0e62-4f53-b347-158ade163929 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:36.950827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3319caf-0e62-4f53-b347-158ade163929 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:36.951312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3319caf-0e62-4f53-b347-158ade163929 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:36.953070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3319caf-0e62-4f53-b347-158ade163929 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:36.953620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3319caf-0e62-4f53-b347-158ade163929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:36.965927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3319caf-0e62-4f53-b347-158ade163929 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:36.996235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 354/692] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:37.001787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3319caf-0e62-4f53-b347-158ade163929 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:37.020359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acc38c-feb3-4879-a289-2f7c7088f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:37.022789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acc38c-feb3-4879-a289-2f7c7088f043 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:37.024428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acc38c-feb3-4879-a289-2f7c7088f043 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:37.025336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acc38c-feb3-4879-a289-2f7c7088f043 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:37.027967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acc38c-feb3-4879-a289-2f7c7088f043 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:37.028854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acc38c-feb3-4879-a289-2f7c7088f043 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:37.032873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acc38c-feb3-4879-a289-2f7c7088f043 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:37.108494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 339/693] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 6 14:08:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:37.113256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acc38c-feb3-4879-a289-2f7c7088f043 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:37.157647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:37.294754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3319caf-0e62-4f53-b347-158ade163929 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:37.341420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acc38c-feb3-4879-a289-2f7c7088f043 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:37.380062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07799b-6b25-4e7f-9054-ef7b60cdace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:37.381379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07799b-6b25-4e7f-9054-ef7b60cdace1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:37.381917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07799b-6b25-4e7f-9054-ef7b60cdace1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:37.382432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07799b-6b25-4e7f-9054-ef7b60cdace1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:37.387981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:37.407372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07799b-6b25-4e7f-9054-ef7b60cdace1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:37.409450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07799b-6b25-4e7f-9054-ef7b60cdace1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:37.418427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475223a4-d8dd-47fd-9403-e34a74059381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:37.419505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475223a4-d8dd-47fd-9403-e34a74059381 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:37.420003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475223a4-d8dd-47fd-9403-e34a74059381 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:37.420507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475223a4-d8dd-47fd-9403-e34a74059381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-dc07799b-6b25-4e7f-9054-ef7b60cdace1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:37.459191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:37.460178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:37.461100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:37.461100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:37.461100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:37.461100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:37.474202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 355/694] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:37.477053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07799b-6b25-4e7f-9054-ef7b60cdace1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:37.583373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475223a4-d8dd-47fd-9403-e34a74059381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:37.585253 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475223a4-d8dd-47fd-9403-e34a74059381 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:37.684426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:37.689743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07799b-6b25-4e7f-9054-ef7b60cdace1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:37.725984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c197f-90ef-4ad8-82d4-d11b7ffb7b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:37.726822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c197f-90ef-4ad8-82d4-d11b7ffb7b63 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:37.727112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c197f-90ef-4ad8-82d4-d11b7ffb7b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:37.727483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c197f-90ef-4ad8-82d4-d11b7ffb7b63 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:37.728238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c197f-90ef-4ad8-82d4-d11b7ffb7b63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:37.728548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c197f-90ef-4ad8-82d4-d11b7ffb7b63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:37.730213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c197f-90ef-4ad8-82d4-d11b7ffb7b63 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:37.750309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 356/695] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:37.751690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c197f-90ef-4ad8-82d4-d11b7ffb7b63 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:37.812404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 340/696] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:37.817625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475223a4-d8dd-47fd-9403-e34a74059381 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:37.869468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:37.872621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c197f-90ef-4ad8-82d4-d11b7ffb7b63 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:37.926989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07095c42-7bec-4928-a6be-3ca16d1e660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:37.928385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07095c42-7bec-4928-a6be-3ca16d1e660e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:37.928890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07095c42-7bec-4928-a6be-3ca16d1e660e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:37.929237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07095c42-7bec-4928-a6be-3ca16d1e660e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:37.963371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07095c42-7bec-4928-a6be-3ca16d1e660e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:37.966145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07095c42-7bec-4928-a6be-3ca16d1e660e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:37.985249 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475223a4-d8dd-47fd-9403-e34a74059381 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-07095c42-7bec-4928-a6be-3ca16d1e660e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:38.021072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:38.022140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:38.023106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:38.023651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 357/697] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 6 14:08:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:38.031924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07095c42-7bec-4928-a6be-3ca16d1e660e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:38.035414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd51a92-f939-4d9a-943e-815159204190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:38.038600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51a92-f939-4d9a-943e-815159204190 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.039138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51a92-f939-4d9a-943e-815159204190 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.040387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51a92-f939-4d9a-943e-815159204190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.077732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd51a92-f939-4d9a-943e-815159204190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:38.079365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd51a92-f939-4d9a-943e-815159204190 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:38.090339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-6bd51a92-f939-4d9a-943e-815159204190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:38.123818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:38.124843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:38.125780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:38.125780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:38.125780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:38.125780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:38.131409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 341/698] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:38.132365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd51a92-f939-4d9a-943e-815159204190 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:38.230386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07095c42-7bec-4928-a6be-3ca16d1e660e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:38.264550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:38.296417 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd51a92-f939-4d9a-943e-815159204190 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:38.310477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206cfc7-e0bb-4fcc-b29c-dfe5170b4044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:38.311173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206cfc7-e0bb-4fcc-b29c-dfe5170b4044 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.311444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206cfc7-e0bb-4fcc-b29c-dfe5170b4044 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.311719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206cfc7-e0bb-4fcc-b29c-dfe5170b4044 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.312581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206cfc7-e0bb-4fcc-b29c-dfe5170b4044 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:38.312893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206cfc7-e0bb-4fcc-b29c-dfe5170b4044 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:38.314409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206cfc7-e0bb-4fcc-b29c-dfe5170b4044 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:38.338330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 358/699] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 6 14:08:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:38.339296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206cfc7-e0bb-4fcc-b29c-dfe5170b4044 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:38.343902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c640772-7910-41c3-b0bd-dbb232e7f2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:38.344584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c640772-7910-41c3-b0bd-dbb232e7f2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.344867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c640772-7910-41c3-b0bd-dbb232e7f2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.345138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c640772-7910-41c3-b0bd-dbb232e7f2f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.345898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c640772-7910-41c3-b0bd-dbb232e7f2f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:38.346230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c640772-7910-41c3-b0bd-dbb232e7f2f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:38.347591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c640772-7910-41c3-b0bd-dbb232e7f2f9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:38.378947 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 342/700] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:38.385760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c640772-7910-41c3-b0bd-dbb232e7f2f9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:38.469324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:38.499625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206cfc7-e0bb-4fcc-b29c-dfe5170b4044 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:38.543011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f74f6-fe88-4544-b857-15966f51b350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:38.543701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f74f6-fe88-4544-b857-15966f51b350 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.543989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f74f6-fe88-4544-b857-15966f51b350 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.544268 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f74f6-fe88-4544-b857-15966f51b350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.546862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c640772-7910-41c3-b0bd-dbb232e7f2f9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:38.601209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47073329-eab8-436b-890e-65cb47295b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:38.602961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47073329-eab8-436b-890e-65cb47295b76 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.603251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47073329-eab8-436b-890e-65cb47295b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.603632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47073329-eab8-436b-890e-65cb47295b76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.606384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:38.612358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47073329-eab8-436b-890e-65cb47295b76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:38.614344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47073329-eab8-436b-890e-65cb47295b76 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-47073329-eab8-436b-890e-65cb47295b76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:38.630346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:38.632452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 343/701] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:38.634896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9ab4-19bb-48c6-9292-7f6dddb1f9ca None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.635761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9ab4-19bb-48c6-9292-7f6dddb1f9ca None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.635761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade9ab4-19bb-48c6-9292-7f6dddb1f9ca None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.637099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 344/702] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:38.638386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1a462-be1c-4f4a-aab4-bf47107d2724 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.638674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1a462-be1c-4f4a-aab4-bf47107d2724 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.638927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1a462-be1c-4f4a-aab4-bf47107d2724 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.640383 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 345/703] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:38.641103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1a462-be1c-4f4a-aab4-bf47107d2724 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:38.674273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f74f6-fe88-4544-b857-15966f51b350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:38.677008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f74f6-fe88-4544-b857-15966f51b350 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:38.769017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1a462-be1c-4f4a-aab4-bf47107d2724 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:38.797424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:38.818111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fb916-00cf-4bcb-a08c-dd4cc0ebf861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:38.818775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fb916-00cf-4bcb-a08c-dd4cc0ebf861 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.819062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fb916-00cf-4bcb-a08c-dd4cc0ebf861 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.819350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fb916-00cf-4bcb-a08c-dd4cc0ebf861 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.820131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727fb916-00cf-4bcb-a08c-dd4cc0ebf861 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:38.820439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727fb916-00cf-4bcb-a08c-dd4cc0ebf861 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:38.822028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727fb916-00cf-4bcb-a08c-dd4cc0ebf861 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:38.833136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 359/704] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:38.835209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54a21f-980c-4117-bd60-18ac0f27cc88 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.835796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54a21f-980c-4117-bd60-18ac0f27cc88 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.836284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54a21f-980c-4117-bd60-18ac0f27cc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.839601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 346/705] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:38.841085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544539a-00ca-4a57-9ed3-4a5e8fc911c6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:38.841371 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544539a-00ca-4a57-9ed3-4a5e8fc911c6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:38.841651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544539a-00ca-4a57-9ed3-4a5e8fc911c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:38.858954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3e54a21f-980c-4117-bd60-18ac0f27cc88 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:38.860282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9544539a-00ca-4a57-9ed3-4a5e8fc911c6 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:38.860560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3e54a21f-980c-4117-bd60-18ac0f27cc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:38.861484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9544539a-00ca-4a57-9ed3-4a5e8fc911c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:38.927759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:38.946562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 347/706] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:38.949781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544539a-00ca-4a57-9ed3-4a5e8fc911c6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:38.949984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 360/707] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:38.950333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54a21f-980c-4117-bd60-18ac0f27cc88 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:39.079348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54a21f-980c-4117-bd60-18ac0f27cc88 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:39.090139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:39.091824 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544539a-00ca-4a57-9ed3-4a5e8fc911c6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:39.133636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac36a4-efab-43a8-a900-37e63545d0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:39.134323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac36a4-efab-43a8-a900-37e63545d0bd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.134813 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac36a4-efab-43a8-a900-37e63545d0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.134996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac36a4-efab-43a8-a900-37e63545d0bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.135847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac36a4-efab-43a8-a900-37e63545d0bd None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=3571f3d3a0de4e9abe55ed84426143a0) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:39.136158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac36a4-efab-43a8-a900-37e63545d0bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:39.137629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac36a4-efab-43a8-a900-37e63545d0bd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:39.143996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 361/708] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:38 2022] GET /identity/v3/role_assignments?user.id=3571f3d3a0de4e9abe55ed84426143a0&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:39.145302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9ebd8-d5cf-46cb-822b-fb415cb1c0b2 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.145588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9ebd8-d5cf-46cb-822b-fb415cb1c0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.145896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9ebd8-d5cf-46cb-822b-fb415cb1c0b2 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.147417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 362/709] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:39.148164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9ebd8-d5cf-46cb-822b-fb415cb1c0b2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:39.157990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e15e0-60cc-4065-978f-7442a92bdcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:39.161421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e15e0-60cc-4065-978f-7442a92bdcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.161421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e15e0-60cc-4065-978f-7442a92bdcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.161421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e15e0-60cc-4065-978f-7442a92bdcb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.176032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e15e0-60cc-4065-978f-7442a92bdcb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:39.178473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e15e0-60cc-4065-978f-7442a92bdcb8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2c9e15e0-60cc-4065-978f-7442a92bdcb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:39.205728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:39.218944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:39.220309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:39.220309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:39.220309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:39.220309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:39.220309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 348/710] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:39.220309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e15e0-60cc-4065-978f-7442a92bdcb8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:39.258767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:39.310696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9ebd8-d5cf-46cb-822b-fb415cb1c0b2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:39.353906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39ec59-5f5d-49f1-952a-85df805f547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:39.354632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39ec59-5f5d-49f1-952a-85df805f547d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.354951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39ec59-5f5d-49f1-952a-85df805f547d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.355202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39ec59-5f5d-49f1-952a-85df805f547d None admin] PATH_INFO: `/v3/services/network` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.355985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39ec59-5f5d-49f1-952a-85df805f547d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:39.357590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39ec59-5f5d-49f1-952a-85df805f547d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:39.360905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e15e0-60cc-4065-978f-7442a92bdcb8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-5c39ec59-5f5d-49f1-952a-85df805f547d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:39.379097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:39.380907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:39.383101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 363/711] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:39 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:39.387677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed162a7-f8f3-485c-bd66-a875219b1776 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.387677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed162a7-f8f3-485c-bd66-a875219b1776 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.387677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed162a7-f8f3-485c-bd66-a875219b1776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.406682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-eed162a7-f8f3-485c-bd66-a875219b1776 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:39.407885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-eed162a7-f8f3-485c-bd66-a875219b1776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:39.411418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f80df3-fb60-4135-b079-a8c037494804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:39.412567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f80df3-fb60-4135-b079-a8c037494804 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.413310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f80df3-fb60-4135-b079-a8c037494804 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.413606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f80df3-fb60-4135-b079-a8c037494804 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.436097 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f80df3-fb60-4135-b079-a8c037494804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d6f80df3-fb60-4135-b079-a8c037494804 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:39.442101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:39.443144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:39.444427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 349/712] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:39.445559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f80df3-fb60-4135-b079-a8c037494804 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:39.453358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:39.517250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 364/713] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:39.521978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed162a7-f8f3-485c-bd66-a875219b1776 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:39.581790 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:39.591219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f80df3-fb60-4135-b079-a8c037494804 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:39.640721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c95068-d74f-4d81-b1c5-d6b5e4723d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:39.641290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c95068-d74f-4d81-b1c5-d6b5e4723d8a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.641578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c95068-d74f-4d81-b1c5-d6b5e4723d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.641907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c95068-d74f-4d81-b1c5-d6b5e4723d8a None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.642653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c95068-d74f-4d81-b1c5-d6b5e4723d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:39.642959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c95068-d74f-4d81-b1c5-d6b5e4723d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:39.644375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c95068-d74f-4d81-b1c5-d6b5e4723d8a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:39.658666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 350/714] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:39.659548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c95068-d74f-4d81-b1c5-d6b5e4723d8a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:39.691864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed162a7-f8f3-485c-bd66-a875219b1776 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:39.738244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9569a2-84eb-4430-b2b0-dcaab4bb2cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:39.739147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9569a2-84eb-4430-b2b0-dcaab4bb2cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.739637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9569a2-84eb-4430-b2b0-dcaab4bb2cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.740017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9569a2-84eb-4430-b2b0-dcaab4bb2cb4 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.740953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9569a2-84eb-4430-b2b0-dcaab4bb2cb4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:39.741269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9569a2-84eb-4430-b2b0-dcaab4bb2cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:39.743836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9569a2-84eb-4430-b2b0-dcaab4bb2cb4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:39.750176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 365/715] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:08:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:39.751051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9569a2-84eb-4430-b2b0-dcaab4bb2cb4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:39.796073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:39.799439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c95068-d74f-4d81-b1c5-d6b5e4723d8a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:39.837392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56667d3d-6807-4e41-ba21-64087411751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:39.838098 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56667d3d-6807-4e41-ba21-64087411751a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.838383 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56667d3d-6807-4e41-ba21-64087411751a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.838668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56667d3d-6807-4e41-ba21-64087411751a None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.839437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56667d3d-6807-4e41-ba21-64087411751a None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:39.839840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56667d3d-6807-4e41-ba21-64087411751a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:39.841181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56667d3d-6807-4e41-ba21-64087411751a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:39.851162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 351/716] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:39.851827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56667d3d-6807-4e41-ba21-64087411751a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:39.884667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:39.905178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9569a2-84eb-4430-b2b0-dcaab4bb2cb4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:39.954388 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd88eb-2bf3-48e5-8ab0-59121ab3b680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:39.955469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd88eb-2bf3-48e5-8ab0-59121ab3b680 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:39.955925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd88eb-2bf3-48e5-8ab0-59121ab3b680 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:39.956353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd88eb-2bf3-48e5-8ab0-59121ab3b680 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:39.957161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd88eb-2bf3-48e5-8ab0-59121ab3b680 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:39.959605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd88eb-2bf3-48e5-8ab0-59121ab3b680 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b5fd88eb-2bf3-48e5-8ab0-59121ab3b680 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:39.975171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:39.982038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:39.983381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 366/717] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Jun 6 14:08:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:39.985609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd88eb-2bf3-48e5-8ab0-59121ab3b680 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:40.013417 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56667d3d-6807-4e41-ba21-64087411751a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:40.062061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2133200-076e-457b-90d6-cc8f2a65e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:40.062906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2133200-076e-457b-90d6-cc8f2a65e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:40.063200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2133200-076e-457b-90d6-cc8f2a65e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:40.063578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2133200-076e-457b-90d6-cc8f2a65e4e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:40.073444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2133200-076e-457b-90d6-cc8f2a65e4e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:40.075093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2133200-076e-457b-90d6-cc8f2a65e4e4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c2133200-076e-457b-90d6-cc8f2a65e4e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:40.092252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:40.095305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:40.097092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:40.097092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:40.097092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:40.097092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:40.097092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:40.097092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:40.097092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:40.097092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:40.097092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 352/718] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:40.099365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2133200-076e-457b-90d6-cc8f2a65e4e4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:40.136053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:40.148076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd88eb-2bf3-48e5-8ab0-59121ab3b680 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:40.211012 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf24a9-cf22-4893-b8e5-027a701db96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:40.212205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf24a9-cf22-4893-b8e5-027a701db96e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:40.212772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf24a9-cf22-4893-b8e5-027a701db96e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:40.213224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf24a9-cf22-4893-b8e5-027a701db96e None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:40.214224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf24a9-cf22-4893-b8e5-027a701db96e None admin] RBAC: Adding query filter params (type=network) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:40.214776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf24a9-cf22-4893-b8e5-027a701db96e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:40.217532 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf24a9-cf22-4893-b8e5-027a701db96e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:40.237750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 367/719] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:08:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:40.237750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf24a9-cf22-4893-b8e5-027a701db96e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:40.240835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:40.283010 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2133200-076e-457b-90d6-cc8f2a65e4e4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:40.353039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e87998-8e01-4191-9655-e65ab0ad0245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:40.354422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e87998-8e01-4191-9655-e65ab0ad0245 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:40.354911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e87998-8e01-4191-9655-e65ab0ad0245 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:40.355376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e87998-8e01-4191-9655-e65ab0ad0245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:40.371040 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e87998-8e01-4191-9655-e65ab0ad0245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:40.373191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e87998-8e01-4191-9655-e65ab0ad0245 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-70e87998-8e01-4191-9655-e65ab0ad0245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:40.398373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:40.400108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:40.401187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:40.401187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:40.401187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:40.401187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:40.403486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 353/720] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:40.405039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e87998-8e01-4191-9655-e65ab0ad0245 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:40.419849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf24a9-cf22-4893-b8e5-027a701db96e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:40.487850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a43e7c-8a4a-4ec2-a253-22ad79d8ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:40.488845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a43e7c-8a4a-4ec2-a253-22ad79d8ec65 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:40.489320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a43e7c-8a4a-4ec2-a253-22ad79d8ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:40.490440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a43e7c-8a4a-4ec2-a253-22ad79d8ec65 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:40.491383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a43e7c-8a4a-4ec2-a253-22ad79d8ec65 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:40.491858 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a43e7c-8a4a-4ec2-a253-22ad79d8ec65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:40.494553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a43e7c-8a4a-4ec2-a253-22ad79d8ec65 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:40.501906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 368/721] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 6 14:08:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:40.503039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a43e7c-8a4a-4ec2-a253-22ad79d8ec65 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:40.551819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:40.620250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e87998-8e01-4191-9655-e65ab0ad0245 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:40.654382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:40.660107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a43e7c-8a4a-4ec2-a253-22ad79d8ec65 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:40.661351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340c1a6-e9eb-412c-8f3b-afbc297d41f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:40.662366 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340c1a6-e9eb-412c-8f3b-afbc297d41f5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:40.662766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340c1a6-e9eb-412c-8f3b-afbc297d41f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:40.663143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340c1a6-e9eb-412c-8f3b-afbc297d41f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:40.665294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340c1a6-e9eb-412c-8f3b-afbc297d41f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:40.666067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340c1a6-e9eb-412c-8f3b-afbc297d41f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:40.667658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340c1a6-e9eb-412c-8f3b-afbc297d41f5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:40.689385 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 354/722] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:40.689385 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340c1a6-e9eb-412c-8f3b-afbc297d41f5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:40.710307 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927514da-8c58-4ece-97fc-68b6515b2ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:40.711461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927514da-8c58-4ece-97fc-68b6515b2ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:40.712755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927514da-8c58-4ece-97fc-68b6515b2ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:40.713211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927514da-8c58-4ece-97fc-68b6515b2ee7 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:40.714160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927514da-8c58-4ece-97fc-68b6515b2ee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:40.714669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927514da-8c58-4ece-97fc-68b6515b2ee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:40.716845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927514da-8c58-4ece-97fc-68b6515b2ee7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:40.735019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 369/723] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:40.736176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927514da-8c58-4ece-97fc-68b6515b2ee7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:40.826870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340c1a6-e9eb-412c-8f3b-afbc297d41f5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:40.863903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:40.875778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420c2e2-a98a-4fda-bd93-7cf9b6f8f8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:40.876577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c2e2-a98a-4fda-bd93-7cf9b6f8f8dc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:40.876938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c2e2-a98a-4fda-bd93-7cf9b6f8f8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:40.877291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c2e2-a98a-4fda-bd93-7cf9b6f8f8dc None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:40.878149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420c2e2-a98a-4fda-bd93-7cf9b6f8f8dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:40.878548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420c2e2-a98a-4fda-bd93-7cf9b6f8f8dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:40.881959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420c2e2-a98a-4fda-bd93-7cf9b6f8f8dc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:40.891569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 355/724] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 6 14:08:40 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:40.893247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420c2e2-a98a-4fda-bd93-7cf9b6f8f8dc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:40.904124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927514da-8c58-4ece-97fc-68b6515b2ee7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:40.969058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e74610-219d-493c-8352-ee8dea808b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:40.971723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e74610-219d-493c-8352-ee8dea808b06 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:40.972376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e74610-219d-493c-8352-ee8dea808b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:40.972883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e74610-219d-493c-8352-ee8dea808b06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:40.974761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e74610-219d-493c-8352-ee8dea808b06 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=0b82a920df4042dd97314350aed4d7a5) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:40.975427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e74610-219d-493c-8352-ee8dea808b06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:40.978050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e74610-219d-493c-8352-ee8dea808b06 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:40.987345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:40.995352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 370/725] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:40 2022] GET /identity/v3/role_assignments?user.id=0b82a920df4042dd97314350aed4d7a5&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:41.062416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420c2e2-a98a-4fda-bd93-7cf9b6f8f8dc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:41.142145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cf53f-bad0-4c7a-a04e-a5bc670f0229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:41.143551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cf53f-bad0-4c7a-a04e-a5bc670f0229 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:41.144054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cf53f-bad0-4c7a-a04e-a5bc670f0229 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:41.144569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cf53f-bad0-4c7a-a04e-a5bc670f0229 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:41.160455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16cf53f-bad0-4c7a-a04e-a5bc670f0229 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:41.163336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16cf53f-bad0-4c7a-a04e-a5bc670f0229 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c16cf53f-bad0-4c7a-a04e-a5bc670f0229 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:41.196176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:41.197616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:08:41.198734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:41.199816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 356/726] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:08:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:41.205498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e74610-219d-493c-8352-ee8dea808b06 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:41.355946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e74610-219d-493c-8352-ee8dea808b06 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:41.408281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f95e5-4baf-4d70-b53f-6acd6a24ae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:41.409932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f95e5-4baf-4d70-b53f-6acd6a24ae5b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:41.410759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f95e5-4baf-4d70-b53f-6acd6a24ae5b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:41.411559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f95e5-4baf-4d70-b53f-6acd6a24ae5b None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:41.413467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f95e5-4baf-4d70-b53f-6acd6a24ae5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:41.414356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f95e5-4baf-4d70-b53f-6acd6a24ae5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:41.417957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f95e5-4baf-4d70-b53f-6acd6a24ae5b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:41.453247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 371/727] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 6 14:08:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:41.462143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cf53f-bad0-4c7a-a04e-a5bc670f0229 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:41.636535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cf53f-bad0-4c7a-a04e-a5bc670f0229 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:41.659120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:41.690115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e1a11-f55a-492f-9766-f0d1b2652b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:41.690781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e1a11-f55a-492f-9766-f0d1b2652b9c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:41.691062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e1a11-f55a-492f-9766-f0d1b2652b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:41.691335 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e1a11-f55a-492f-9766-f0d1b2652b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:41.801114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e1a11-f55a-492f-9766-f0d1b2652b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:41.802632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e1a11-f55a-492f-9766-f0d1b2652b9c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:41.948114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 357/728] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:41.956115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f95e5-4baf-4d70-b53f-6acd6a24ae5b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:41.960159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:42.144902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f95e5-4baf-4d70-b53f-6acd6a24ae5b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:42.206428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86014a-87a0-42fa-93cf-bf9b15440f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:42.207727 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86014a-87a0-42fa-93cf-bf9b15440f9e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:42.208237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86014a-87a0-42fa-93cf-bf9b15440f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:42.208737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86014a-87a0-42fa-93cf-bf9b15440f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:42.232973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86014a-87a0-42fa-93cf-bf9b15440f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:42.234982 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86014a-87a0-42fa-93cf-bf9b15440f9e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-9c86014a-87a0-42fa-93cf-bf9b15440f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:42.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:42.261985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:42.263432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:42.263432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:42.263432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:42.263432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:42.265687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 372/729] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:42.268071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e1a11-f55a-492f-9766-f0d1b2652b9c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:42.320553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:42.476594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e1a11-f55a-492f-9766-f0d1b2652b9c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:42.526228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e976-d7a2-4be1-a7b2-92959ee9f24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:42.527320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e976-d7a2-4be1-a7b2-92959ee9f24a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:42.527826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e976-d7a2-4be1-a7b2-92959ee9f24a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:42.528303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e976-d7a2-4be1-a7b2-92959ee9f24a None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:42.529263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93e976-d7a2-4be1-a7b2-92959ee9f24a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:42.529810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93e976-d7a2-4be1-a7b2-92959ee9f24a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:42.531439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93e976-d7a2-4be1-a7b2-92959ee9f24a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:42.545732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46949687-0c9f-4e59-b20e-6b855a9247d9 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:42.545732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46949687-0c9f-4e59-b20e-6b855a9247d9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:42.545732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46949687-0c9f-4e59-b20e-6b855a9247d9 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:42.545732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 373/730] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:42.552401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648055f4-cd24-4ef1-8d9b-8f05ba702430 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:42.552902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648055f4-cd24-4ef1-8d9b-8f05ba702430 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:42.553659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648055f4-cd24-4ef1-8d9b-8f05ba702430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:42.555936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 358/731] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:42.561793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e976-d7a2-4be1-a7b2-92959ee9f24a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:42.590453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-648055f4-cd24-4ef1-8d9b-8f05ba702430 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:42.592785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-648055f4-cd24-4ef1-8d9b-8f05ba702430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:42.737624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:42.751283 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:42.788021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 374/732] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:42.818390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e976-d7a2-4be1-a7b2-92959ee9f24a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:42.829895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648055f4-cd24-4ef1-8d9b-8f05ba702430 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:42.874126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee85b6-e3c2-4d38-b6af-5242c54238ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:42.874917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee85b6-e3c2-4d38-b6af-5242c54238ca None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:42.875210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee85b6-e3c2-4d38-b6af-5242c54238ca None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:42.875471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee85b6-e3c2-4d38-b6af-5242c54238ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:42.885206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee85b6-e3c2-4d38-b6af-5242c54238ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:42.886799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee85b6-e3c2-4d38-b6af-5242c54238ca None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-08ee85b6-e3c2-4d38-b6af-5242c54238ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:42.904258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:42.906026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:42.907771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:42.907771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:42.907771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:42.907771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:42.907771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:42.907771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:42.907771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:42.907771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:42.909990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 359/733] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:42.911818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee85b6-e3c2-4d38-b6af-5242c54238ca None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:43.042282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee85b6-e3c2-4d38-b6af-5242c54238ca None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:43.119064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b040620-f67e-462e-be9a-ffca865536d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:43.120307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b040620-f67e-462e-be9a-ffca865536d6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:43.120819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b040620-f67e-462e-be9a-ffca865536d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:43.121324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b040620-f67e-462e-be9a-ffca865536d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:43.122931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b040620-f67e-462e-be9a-ffca865536d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:43.123511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b040620-f67e-462e-be9a-ffca865536d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:43.125697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b040620-f67e-462e-be9a-ffca865536d6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:43.128675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:43.138271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648055f4-cd24-4ef1-8d9b-8f05ba702430 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:43.164512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 360/734] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:43.172406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b040620-f67e-462e-be9a-ffca865536d6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:43.213410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af1ec6-4330-443b-b439-4f1a3b95f058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:43.214265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1ec6-4330-443b-b439-4f1a3b95f058 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:43.214557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1ec6-4330-443b-b439-4f1a3b95f058 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:43.214832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1ec6-4330-443b-b439-4f1a3b95f058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:43.226579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af1ec6-4330-443b-b439-4f1a3b95f058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d4af1ec6-4330-443b-b439-4f1a3b95f058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:43.235249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:43.236895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:43.237810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 375/735] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:43.238848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88982-0cda-4ae3-a509-8be0fd21b4c9 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:43.238848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88982-0cda-4ae3-a509-8be0fd21b4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:43.238848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88982-0cda-4ae3-a509-8be0fd21b4c9 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:43.244147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 376/736] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:43.245120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88982-0cda-4ae3-a509-8be0fd21b4c9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:43.299428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:43.363971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88982-0cda-4ae3-a509-8be0fd21b4c9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:43.379172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b040620-f67e-462e-be9a-ffca865536d6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:43.405785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a9e18-0d5e-44c7-a78d-490607a92667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:43.406474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a9e18-0d5e-44c7-a78d-490607a92667 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:43.406754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a9e18-0d5e-44c7-a78d-490607a92667 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:43.407036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a9e18-0d5e-44c7-a78d-490607a92667 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:43.407808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a9e18-0d5e-44c7-a78d-490607a92667 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:43.408120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a9e18-0d5e-44c7-a78d-490607a92667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:43.409588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a9e18-0d5e-44c7-a78d-490607a92667 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:43.422521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 377/737] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:43.424091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a6bd5-ee0b-4205-9176-317d26ba85e4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:43.424391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a6bd5-ee0b-4205-9176-317d26ba85e4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:43.424673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a6bd5-ee0b-4205-9176-317d26ba85e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:43.442204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45cc71-01f9-4954-8e3b-3a7480ea7508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:43.443516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45cc71-01f9-4954-8e3b-3a7480ea7508 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:43.444160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45cc71-01f9-4954-8e3b-3a7480ea7508 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:43.444748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45cc71-01f9-4954-8e3b-3a7480ea7508 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:43.445832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e29a6bd5-ee0b-4205-9176-317d26ba85e4 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:43.446138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45cc71-01f9-4954-8e3b-3a7480ea7508 None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=bb0da8665c1b44f79611129ee1337bdc) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:43.446670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45cc71-01f9-4954-8e3b-3a7480ea7508 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:43.447380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e29a6bd5-ee0b-4205-9176-317d26ba85e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:43.448639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45cc71-01f9-4954-8e3b-3a7480ea7508 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:43.462307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 361/738] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:43 2022] GET /identity/v3/role_assignments?user.id=bb0da8665c1b44f79611129ee1337bdc&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:43.463806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45cc71-01f9-4954-8e3b-3a7480ea7508 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:43.560331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:43.571849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 378/739] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:43.620421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45cc71-01f9-4954-8e3b-3a7480ea7508 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:43.623081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a6bd5-ee0b-4205-9176-317d26ba85e4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:43.674097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:43.748760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a6bd5-ee0b-4205-9176-317d26ba85e4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:43.755061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea3e00-828b-4143-86a5-d2a125754fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:43.756450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea3e00-828b-4143-86a5-d2a125754fdb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:43.756941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea3e00-828b-4143-86a5-d2a125754fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:43.757438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea3e00-828b-4143-86a5-d2a125754fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:43.793220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2cb19-8512-4bf1-a650-979f8cec86b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:43.794006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2cb19-8512-4bf1-a650-979f8cec86b4 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:43.794293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2cb19-8512-4bf1-a650-979f8cec86b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:43.794575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2cb19-8512-4bf1-a650-979f8cec86b4 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:43.795375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2cb19-8512-4bf1-a650-979f8cec86b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:43.796823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2cb19-8512-4bf1-a650-979f8cec86b4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:43.803757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea3e00-828b-4143-86a5-d2a125754fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:43.810163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea3e00-828b-4143-86a5-d2a125754fdb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:43.825853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 379/740] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:08:43 2022] POST /identity/v3/services => generated 240 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-3cea3e00-828b-4143-86a5-d2a125754fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:43.859712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:43.861852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:43.863858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:43.863858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:43.863858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:43.863858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:43.866970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 362/741] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:43.874500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2cb19-8512-4bf1-a650-979f8cec86b4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:44.115756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:44.151541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2cb19-8512-4bf1-a650-979f8cec86b4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:44.236633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20bf90-39b5-4fa4-bffa-5663adabeeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:44.237748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20bf90-39b5-4fa4-bffa-5663adabeeab None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:44.238226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20bf90-39b5-4fa4-bffa-5663adabeeab None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:44.238743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20bf90-39b5-4fa4-bffa-5663adabeeab None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:44.240112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20bf90-39b5-4fa4-bffa-5663adabeeab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:44.240626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20bf90-39b5-4fa4-bffa-5663adabeeab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:44.243673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20bf90-39b5-4fa4-bffa-5663adabeeab None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:44.269280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 380/742] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:44.279389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea3e00-828b-4143-86a5-d2a125754fdb None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:44.489091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea3e00-828b-4143-86a5-d2a125754fdb None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:44.515915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:44.516170 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793ef74-d61f-42ce-af00-ff5d1e601d0f None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:44.516506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793ef74-d61f-42ce-af00-ff5d1e601d0f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:44.516879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793ef74-d61f-42ce-af00-ff5d1e601d0f None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:44.519281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 381/743] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:44.531203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652682c-3720-4f7a-b940-28d3e4bdbc1a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:44.531689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652682c-3720-4f7a-b940-28d3e4bdbc1a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:44.532148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652682c-3720-4f7a-b940-28d3e4bdbc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:44.590008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6652682c-3720-4f7a-b940-28d3e4bdbc1a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:44.594338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6652682c-3720-4f7a-b940-28d3e4bdbc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:44.602915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ebe8a-6300-4501-9c89-2e4204ebc5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:44.605060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ebe8a-6300-4501-9c89-2e4204ebc5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:44.605963 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ebe8a-6300-4501-9c89-2e4204ebc5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:44.606838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ebe8a-6300-4501-9c89-2e4204ebc5a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:44.639075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ebe8a-6300-4501-9c89-2e4204ebc5a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:44.640653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ebe8a-6300-4501-9c89-2e4204ebc5a1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-6d8ebe8a-6300-4501-9c89-2e4204ebc5a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:44.661411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:44.663455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:44.667001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 363/744] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:44.672343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ebe8a-6300-4501-9c89-2e4204ebc5a1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:44.840738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 382/745] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:44 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 14:08:44.881533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652682c-3720-4f7a-b940-28d3e4bdbc1a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:45.007493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ebe8a-6300-4501-9c89-2e4204ebc5a1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:45.040307 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:45.061827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db2e70-441d-4202-85ec-a70dd35f200a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:45.062856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2e70-441d-4202-85ec-a70dd35f200a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:45.063276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652682c-3720-4f7a-b940-28d3e4bdbc1a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:45.063457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2e70-441d-4202-85ec-a70dd35f200a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:45.063938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2e70-441d-4202-85ec-a70dd35f200a None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:45.065413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db2e70-441d-4202-85ec-a70dd35f200a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:45.067030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db2e70-441d-4202-85ec-a70dd35f200a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:45.068846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db2e70-441d-4202-85ec-a70dd35f200a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:45.096937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 364/746] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 6 14:08:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:45.099778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338a73b-0e44-41ae-ac37-87e220e4b64e None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:45.100275 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338a73b-0e44-41ae-ac37-87e220e4b64e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:45.100738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338a73b-0e44-41ae-ac37-87e220e4b64e None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:45.104334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 365/747] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:45.105428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338a73b-0e44-41ae-ac37-87e220e4b64e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:45.108510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbf2af-1d5c-41ba-9d08-16754da68eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:45.109626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbf2af-1d5c-41ba-9d08-16754da68eea None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:45.110960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbf2af-1d5c-41ba-9d08-16754da68eea None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:45.110960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbf2af-1d5c-41ba-9d08-16754da68eea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:45.126201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbf2af-1d5c-41ba-9d08-16754da68eea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-fbbbf2af-1d5c-41ba-9d08-16754da68eea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:45.135707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:45.136973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:45.138095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:45.140154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 383/748] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:45.141937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946f5ee-a2a8-4fb4-a7b7-d5b81721790c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:45.142771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946f5ee-a2a8-4fb4-a7b7-d5b81721790c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:45.142771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946f5ee-a2a8-4fb4-a7b7-d5b81721790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:45.146427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:45.163140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2946f5ee-a2a8-4fb4-a7b7-d5b81721790c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:45.164950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2946f5ee-a2a8-4fb4-a7b7-d5b81721790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:45.256076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 384/749] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:45 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 14:08:45.257321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946f5ee-a2a8-4fb4-a7b7-d5b81721790c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:45.361343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338a73b-0e44-41ae-ac37-87e220e4b64e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:45.372948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:45.415827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946f5ee-a2a8-4fb4-a7b7-d5b81721790c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:45.427349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b1276-fb36-48b3-9036-aea15af7e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:45.428008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b1276-fb36-48b3-9036-aea15af7e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:45.428293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b1276-fb36-48b3-9036-aea15af7e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:45.428572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b1276-fb36-48b3-9036-aea15af7e5e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:45.506682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343bfe1-9046-4f0a-b5b5-61717d30bab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:45.509063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343bfe1-9046-4f0a-b5b5-61717d30bab3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:45.510548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343bfe1-9046-4f0a-b5b5-61717d30bab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:45.511594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343bfe1-9046-4f0a-b5b5-61717d30bab3 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:45.513484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343bfe1-9046-4f0a-b5b5-61717d30bab3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:45.514171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343bfe1-9046-4f0a-b5b5-61717d30bab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:45.515643 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343bfe1-9046-4f0a-b5b5-61717d30bab3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:45.525603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b1276-fb36-48b3-9036-aea15af7e5e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:45.527206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b1276-fb36-48b3-9036-aea15af7e5e4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:45.541769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 385/750] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:45.541769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343bfe1-9046-4f0a-b5b5-61717d30bab3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:45.587771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:45.687072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343bfe1-9046-4f0a-b5b5-61717d30bab3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:45.728652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bb1ec-ec6c-4613-a951-4cf4f8446631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:45.729437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bb1ec-ec6c-4613-a951-4cf4f8446631 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:45.729825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bb1ec-ec6c-4613-a951-4cf4f8446631 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:45.730167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bb1ec-ec6c-4613-a951-4cf4f8446631 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:45.730974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bb1ec-ec6c-4613-a951-4cf4f8446631 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:45.732593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bb1ec-ec6c-4613-a951-4cf4f8446631 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:45.736987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:45.758386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 386/751] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:08:45 2022] POST /identity/v3/services => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:08:45.759610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bb1ec-ec6c-4613-a951-4cf4f8446631 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:45.782576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 366/752] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:45 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 14:08:45.792602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b1276-fb36-48b3-9036-aea15af7e5e4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:45.979367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bb1ec-ec6c-4613-a951-4cf4f8446631 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:46.025575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950e532-06d4-40b4-87e9-c068fda3def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:46.026518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950e532-06d4-40b4-87e9-c068fda3def4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:46.026805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950e532-06d4-40b4-87e9-c068fda3def4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:46.027674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950e532-06d4-40b4-87e9-c068fda3def4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:46.042438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950e532-06d4-40b4-87e9-c068fda3def4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:46.043992 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950e532-06d4-40b4-87e9-c068fda3def4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f950e532-06d4-40b4-87e9-c068fda3def4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:46.066444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:46.068245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:46.070094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:46.070094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:46.070094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:46.070094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:46.070094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 387/753] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:46.080965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950e532-06d4-40b4-87e9-c068fda3def4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:46.081692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b1276-fb36-48b3-9036-aea15af7e5e4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:46.115028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:46.127009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79519912-fdb2-4e85-aad9-ff8f6c4cef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:46.127816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79519912-fdb2-4e85-aad9-ff8f6c4cef00 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:46.128100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79519912-fdb2-4e85-aad9-ff8f6c4cef00 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:46.128336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79519912-fdb2-4e85-aad9-ff8f6c4cef00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:46.140200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79519912-fdb2-4e85-aad9-ff8f6c4cef00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:46.141674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79519912-fdb2-4e85-aad9-ff8f6c4cef00 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-79519912-fdb2-4e85-aad9-ff8f6c4cef00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:46.160753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:46.162626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:46.164460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:46.164460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:46.164460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:46.164460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:46.168590 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 367/754] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:46.170059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79519912-fdb2-4e85-aad9-ff8f6c4cef00 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:46.191681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:46.336350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950e532-06d4-40b4-87e9-c068fda3def4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:46.340420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79519912-fdb2-4e85-aad9-ff8f6c4cef00 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:46.415132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65a397-8de0-4f85-96b2-113ebbb17e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:46.416891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65a397-8de0-4f85-96b2-113ebbb17e4b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:46.417696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65a397-8de0-4f85-96b2-113ebbb17e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:46.418559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65a397-8de0-4f85-96b2-113ebbb17e4b None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:46.420238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65a397-8de0-4f85-96b2-113ebbb17e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:46.421098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65a397-8de0-4f85-96b2-113ebbb17e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:46.424094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65a397-8de0-4f85-96b2-113ebbb17e4b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:46.428102 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20dbd-b9a0-4ee3-af40-a6757138955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:46.429308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20dbd-b9a0-4ee3-af40-a6757138955c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:46.430920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20dbd-b9a0-4ee3-af40-a6757138955c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:46.432857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20dbd-b9a0-4ee3-af40-a6757138955c None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:46.434393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20dbd-b9a0-4ee3-af40-a6757138955c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:46.434966 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20dbd-b9a0-4ee3-af40-a6757138955c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:46.437609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20dbd-b9a0-4ee3-af40-a6757138955c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:46.452328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 388/755] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:46.460352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65a397-8de0-4f85-96b2-113ebbb17e4b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:46.477921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 368/756] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:46.481212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20dbd-b9a0-4ee3-af40-a6757138955c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:46.596188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:46.674391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65a397-8de0-4f85-96b2-113ebbb17e4b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:46.677816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:46.702688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20dbd-b9a0-4ee3-af40-a6757138955c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:46.722707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c550e6-83a4-43d9-9c4c-2ef3438b36a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:46.723561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c550e6-83a4-43d9-9c4c-2ef3438b36a9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:46.723915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c550e6-83a4-43d9-9c4c-2ef3438b36a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:46.724074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c550e6-83a4-43d9-9c4c-2ef3438b36a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:46.734629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c550e6-83a4-43d9-9c4c-2ef3438b36a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:46.736712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c550e6-83a4-43d9-9c4c-2ef3438b36a9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b7c550e6-83a4-43d9-9c4c-2ef3438b36a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:46.756131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:46.758060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:46.760074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:46.762147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 389/757] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 6 14:08:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:46.764004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89632198-c8b6-4b09-bf00-cfd36cd43991 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:46.764299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89632198-c8b6-4b09-bf00-cfd36cd43991 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:46.764587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89632198-c8b6-4b09-bf00-cfd36cd43991 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:46.766580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 390/758] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:46.767729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89632198-c8b6-4b09-bf00-cfd36cd43991 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:46.789790 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ebfe8-ff31-42e0-9e61-c7e1896728fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:46.791739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ebfe8-ff31-42e0-9e61-c7e1896728fb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:46.792591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ebfe8-ff31-42e0-9e61-c7e1896728fb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:46.793548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ebfe8-ff31-42e0-9e61-c7e1896728fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:46.808242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ebfe8-ff31-42e0-9e61-c7e1896728fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:46.811624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ebfe8-ff31-42e0-9e61-c7e1896728fb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c53ebfe8-ff31-42e0-9e61-c7e1896728fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:46.843570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:46.845315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:46.847226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:46.847226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:46.847226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:46.847226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:46.847226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:46.847226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:46.847226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:46.847226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:46.853374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 369/759] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:46.856487 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd948-4372-446f-b76c-a2ba41325803 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:46.857404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd948-4372-446f-b76c-a2ba41325803 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:46.858246 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd948-4372-446f-b76c-a2ba41325803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:46.890924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-63cfd948-4372-446f-b76c-a2ba41325803 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:46.893490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-63cfd948-4372-446f-b76c-a2ba41325803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:46.893688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89632198-c8b6-4b09-bf00-cfd36cd43991 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:46.938835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b703a-d75b-484a-8ba2-547fbf61ede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:46.939530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b703a-d75b-484a-8ba2-547fbf61ede4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:46.939817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b703a-d75b-484a-8ba2-547fbf61ede4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:46.940094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b703a-d75b-484a-8ba2-547fbf61ede4 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:46.940964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b703a-d75b-484a-8ba2-547fbf61ede4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:46.941282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b703a-d75b-484a-8ba2-547fbf61ede4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:46.943138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b703a-d75b-484a-8ba2-547fbf61ede4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:46.969972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 391/760] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 6 14:08:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:46.972088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b703a-d75b-484a-8ba2-547fbf61ede4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:47.005244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:47.040602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 370/761] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:08:47.042630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd948-4372-446f-b76c-a2ba41325803 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:47.120226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b703a-d75b-484a-8ba2-547fbf61ede4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:47.128078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:47.163683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59603e72-4867-45e6-ab30-564d226e1a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:47.164518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59603e72-4867-45e6-ab30-564d226e1a72 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:47.165401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59603e72-4867-45e6-ab30-564d226e1a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:47.165797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59603e72-4867-45e6-ab30-564d226e1a72 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:47.166828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59603e72-4867-45e6-ab30-564d226e1a72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:47.167094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59603e72-4867-45e6-ab30-564d226e1a72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:47.168702 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59603e72-4867-45e6-ab30-564d226e1a72 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:47.182736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 392/762] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:47.183727 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59603e72-4867-45e6-ab30-564d226e1a72 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:47.280185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd948-4372-446f-b76c-a2ba41325803 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:47.283722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:47.304823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59603e72-4867-45e6-ab30-564d226e1a72 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:47.345546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce8cc7-c870-4e56-a46c-e3cec3aed4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:47.346499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce8cc7-c870-4e56-a46c-e3cec3aed4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:47.346856 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce8cc7-c870-4e56-a46c-e3cec3aed4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:47.347256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce8cc7-c870-4e56-a46c-e3cec3aed4a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:47.357122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ce4b0-b8a1-472e-88b5-b700ac7503bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:47.358946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ce4b0-b8a1-472e-88b5-b700ac7503bb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:47.359982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ce4b0-b8a1-472e-88b5-b700ac7503bb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:47.360742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ce4b0-b8a1-472e-88b5-b700ac7503bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:47.360905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce8cc7-c870-4e56-a46c-e3cec3aed4a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a1ce8cc7-c870-4e56-a46c-e3cec3aed4a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:47.370853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:47.372744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 393/763] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:47.375080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce8cc7-c870-4e56-a46c-e3cec3aed4a5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:47.390344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ce4b0-b8a1-472e-88b5-b700ac7503bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:47.394017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ce4b0-b8a1-472e-88b5-b700ac7503bb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d23ce4b0-b8a1-472e-88b5-b700ac7503bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:47.421786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:47.422775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:47.423695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:47.423695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:47.423695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:47.423695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:47.426164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 371/764] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:47.427526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ce4b0-b8a1-472e-88b5-b700ac7503bb None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:47.528474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce8cc7-c870-4e56-a46c-e3cec3aed4a5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:47.571945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:47.580389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbc8a5-65ca-4881-ae81-8cc8d687420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:47.581087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc8a5-65ca-4881-ae81-8cc8d687420c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:47.581375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc8a5-65ca-4881-ae81-8cc8d687420c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:47.581653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc8a5-65ca-4881-ae81-8cc8d687420c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:47.582586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbc8a5-65ca-4881-ae81-8cc8d687420c None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=3571f3d3a0de4e9abe55ed84426143a0) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:47.582904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbc8a5-65ca-4881-ae81-8cc8d687420c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:47.584391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbc8a5-65ca-4881-ae81-8cc8d687420c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:47.592730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 394/765] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:47 2022] GET /identity/v3/role_assignments?user.id=3571f3d3a0de4e9abe55ed84426143a0&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:47.593567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbc8a5-65ca-4881-ae81-8cc8d687420c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:47.606862 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:47.615282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ce4b0-b8a1-472e-88b5-b700ac7503bb None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:47.685095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42ace8-4196-4472-9159-34d4eec78362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:47.685900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42ace8-4196-4472-9159-34d4eec78362 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:47.686210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42ace8-4196-4472-9159-34d4eec78362 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:47.686668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42ace8-4196-4472-9159-34d4eec78362 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:47.687748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42ace8-4196-4472-9159-34d4eec78362 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:47.688183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42ace8-4196-4472-9159-34d4eec78362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:47.689747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42ace8-4196-4472-9159-34d4eec78362 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:47.709857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 372/766] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:47.714342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88987ef8-5c3e-4c0d-99ce-3bf60783a2e1 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:47.715321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88987ef8-5c3e-4c0d-99ce-3bf60783a2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:47.716172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88987ef8-5c3e-4c0d-99ce-3bf60783a2e1 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:47.719780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 373/767] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:47.721776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88987ef8-5c3e-4c0d-99ce-3bf60783a2e1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:47.775955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbc8a5-65ca-4881-ae81-8cc8d687420c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:47.831533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ac2a3-fd34-4156-8e64-6378a047e1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:47.832331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ac2a3-fd34-4156-8e64-6378a047e1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:47.832806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ac2a3-fd34-4156-8e64-6378a047e1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:47.833143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ac2a3-fd34-4156-8e64-6378a047e1d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:47.834170 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ac2a3-fd34-4156-8e64-6378a047e1d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:47.834343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ac2a3-fd34-4156-8e64-6378a047e1d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:47.835767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ac2a3-fd34-4156-8e64-6378a047e1d8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:47.854250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 395/768] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:47.856088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2ee11-9b3f-4e83-9ac2-27b95b30bbde None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:47.856363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2ee11-9b3f-4e83-9ac2-27b95b30bbde None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:47.856648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2ee11-9b3f-4e83-9ac2-27b95b30bbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:47.878119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5ac2ee11-9b3f-4e83-9ac2-27b95b30bbde None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:47.879535 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5ac2ee11-9b3f-4e83-9ac2-27b95b30bbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:47.933922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88987ef8-5c3e-4c0d-99ce-3bf60783a2e1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:47.955110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:47.965065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 396/769] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:08:47.966091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2ee11-9b3f-4e83-9ac2-27b95b30bbde None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:48.003924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026300f-f7c9-4ef0-b17e-b1008905e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:48.005247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026300f-f7c9-4ef0-b17e-b1008905e623 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:48.005796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026300f-f7c9-4ef0-b17e-b1008905e623 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:48.006281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026300f-f7c9-4ef0-b17e-b1008905e623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:48.020270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026300f-f7c9-4ef0-b17e-b1008905e623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:48.022321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026300f-f7c9-4ef0-b17e-b1008905e623 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:48.026983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8026300f-f7c9-4ef0-b17e-b1008905e623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:48.046209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:48.047219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:48.048126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:48.048126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:48.048126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:48.048126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:48.050628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 374/770] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:48.051844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026300f-f7c9-4ef0-b17e-b1008905e623 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:48.166168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2ee11-9b3f-4e83-9ac2-27b95b30bbde None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:48.170571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026300f-f7c9-4ef0-b17e-b1008905e623 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:48.209402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa5ebe-63f3-441b-bc37-cbd92b3cd2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:48.210603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa5ebe-63f3-441b-bc37-cbd92b3cd2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:48.211106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa5ebe-63f3-441b-bc37-cbd92b3cd2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:48.211725 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa5ebe-63f3-441b-bc37-cbd92b3cd2a3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:48.212620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa5ebe-63f3-441b-bc37-cbd92b3cd2a3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:48.214372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa5ebe-63f3-441b-bc37-cbd92b3cd2a3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-abfa5ebe-63f3-441b-bc37-cbd92b3cd2a3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:48.229452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:48.230565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 14:08:48.231433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:48.232192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328addd-c95f-4d8e-8b83-ad075d242f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:48.232844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328addd-c95f-4d8e-8b83-ad075d242f6d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:48.232844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328addd-c95f-4d8e-8b83-ad075d242f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:48.233335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328addd-c95f-4d8e-8b83-ad075d242f6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:48.235279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 375/771] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:48 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:48.236422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa5ebe-63f3-441b-bc37-cbd92b3cd2a3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:48.248019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328addd-c95f-4d8e-8b83-ad075d242f6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:48.250823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328addd-c95f-4d8e-8b83-ad075d242f6d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b328addd-c95f-4d8e-8b83-ad075d242f6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:48.279861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:48.280998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:48.281941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:48.281941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:48.281941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:48.281941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:48.281941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:48.281941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:48.281941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:48.281941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:48.282860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 397/772] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:48.284375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328addd-c95f-4d8e-8b83-ad075d242f6d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:48.334860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:48.361218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:48.391016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa5ebe-63f3-441b-bc37-cbd92b3cd2a3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:48.426380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328addd-c95f-4d8e-8b83-ad075d242f6d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:48.432980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1928a-a7f1-4f6d-ab8c-6af8415cd80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:48.433994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1928a-a7f1-4f6d-ab8c-6af8415cd80c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:48.434431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1928a-a7f1-4f6d-ab8c-6af8415cd80c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:48.434855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1928a-a7f1-4f6d-ab8c-6af8415cd80c None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:48.435871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1928a-a7f1-4f6d-ab8c-6af8415cd80c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:48.436366 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1928a-a7f1-4f6d-ab8c-6af8415cd80c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:48.437984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1928a-a7f1-4f6d-ab8c-6af8415cd80c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:48.452321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 376/773] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:48.453426 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1928a-a7f1-4f6d-ab8c-6af8415cd80c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:48.481444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319747-5223-4882-9ab7-f803663680c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:48.482128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319747-5223-4882-9ab7-f803663680c6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:48.482419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319747-5223-4882-9ab7-f803663680c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:48.482700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319747-5223-4882-9ab7-f803663680c6 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:48.483427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81319747-5223-4882-9ab7-f803663680c6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:48.483791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81319747-5223-4882-9ab7-f803663680c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:48.485681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81319747-5223-4882-9ab7-f803663680c6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:48.493291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 398/774] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:08:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:48.494883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319747-5223-4882-9ab7-f803663680c6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:48.583538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1928a-a7f1-4f6d-ab8c-6af8415cd80c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:48.609941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:48.627590 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e7af0-f231-44d6-9897-d6a497a08516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:48.629102 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e7af0-f231-44d6-9897-d6a497a08516 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:48.629541 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e7af0-f231-44d6-9897-d6a497a08516 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:48.630115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e7af0-f231-44d6-9897-d6a497a08516 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:48.631070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e7af0-f231-44d6-9897-d6a497a08516 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:48.631555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e7af0-f231-44d6-9897-d6a497a08516 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:48.633172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e7af0-f231-44d6-9897-d6a497a08516 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:48.647361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319747-5223-4882-9ab7-f803663680c6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:48.648389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 377/775] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:48.649548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e7af0-f231-44d6-9897-d6a497a08516 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:48.685698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:48.712570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cc52f-9dd9-4e5a-b83f-6bc0418151e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:48.714209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc52f-9dd9-4e5a-b83f-6bc0418151e7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:48.714735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc52f-9dd9-4e5a-b83f-6bc0418151e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:48.715239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc52f-9dd9-4e5a-b83f-6bc0418151e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:48.730905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cc52f-9dd9-4e5a-b83f-6bc0418151e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:48.733943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cc52f-9dd9-4e5a-b83f-6bc0418151e7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7d6cc52f-9dd9-4e5a-b83f-6bc0418151e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:48.763894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:48.765113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:08:48.766234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:48.772174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 399/776] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:08:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:48.773698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cc52f-9dd9-4e5a-b83f-6bc0418151e7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:48.779734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e7af0-f231-44d6-9897-d6a497a08516 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:48.824788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882b202-5fbe-40bb-b8ea-33f88eaf55c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:48.825954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882b202-5fbe-40bb-b8ea-33f88eaf55c9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:48.826409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882b202-5fbe-40bb-b8ea-33f88eaf55c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:48.826833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882b202-5fbe-40bb-b8ea-33f88eaf55c9 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:48.827699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882b202-5fbe-40bb-b8ea-33f88eaf55c9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:48.828157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882b202-5fbe-40bb-b8ea-33f88eaf55c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:48.829868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882b202-5fbe-40bb-b8ea-33f88eaf55c9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:48.836193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 378/777] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:08:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:48.837868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882b202-5fbe-40bb-b8ea-33f88eaf55c9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:48.892006 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:48.956647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882b202-5fbe-40bb-b8ea-33f88eaf55c9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:48.963469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cc52f-9dd9-4e5a-b83f-6bc0418151e7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:48.998567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8be0df-8133-44f4-88d4-8c178d4d2e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:48.999559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8be0df-8133-44f4-88d4-8c178d4d2e3c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.000030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8be0df-8133-44f4-88d4-8c178d4d2e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.000476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8be0df-8133-44f4-88d4-8c178d4d2e3c None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.001721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8be0df-8133-44f4-88d4-8c178d4d2e3c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:49.002231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8be0df-8133-44f4-88d4-8c178d4d2e3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:49.003873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8be0df-8133-44f4-88d4-8c178d4d2e3c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:49.020456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924bc9-6853-4b82-8d0e-9409468da3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:49.022318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924bc9-6853-4b82-8d0e-9409468da3ad None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.022318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924bc9-6853-4b82-8d0e-9409468da3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.022739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924bc9-6853-4b82-8d0e-9409468da3ad None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/0b82a920df4042dd97314350aed4d7a5/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.025940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c924bc9-6853-4b82-8d0e-9409468da3ad None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=0b82a920df4042dd97314350aed4d7a5, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:49.027509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 379/778] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 6 14:08:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:49.028624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8be0df-8133-44f4-88d4-8c178d4d2e3c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:49.033671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c924bc9-6853-4b82-8d0e-9409468da3ad None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:49.053335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 400/779] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:08:48 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/0b82a920df4042dd97314350aed4d7a5/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:08:49.054826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924bc9-6853-4b82-8d0e-9409468da3ad None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:49.120493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:49.157109 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8be0df-8133-44f4-88d4-8c178d4d2e3c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:49.159221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:49.203565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effab27-4e39-4c46-9724-769794bc8ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:49.204581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effab27-4e39-4c46-9724-769794bc8ead None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.205018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effab27-4e39-4c46-9724-769794bc8ead None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.205449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effab27-4e39-4c46-9724-769794bc8ead None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.206453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effab27-4e39-4c46-9724-769794bc8ead None admin] RBAC: Adding query filter params (interface=public, service_id=1da0c96a58434905a43f7015a7a7cb0b, region_id=RegionOne) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:49.206922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effab27-4e39-4c46-9724-769794bc8ead None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:49.207295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924bc9-6853-4b82-8d0e-9409468da3ad None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:49.208669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effab27-4e39-4c46-9724-769794bc8ead None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:49.219443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 380/780] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 6 14:08:49 2022] GET /identity/v3/endpoints?service_id=1da0c96a58434905a43f7015a7a7cb0b&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:49.228112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effab27-4e39-4c46-9724-769794bc8ead None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:49.251064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22925b5c-cb19-4362-af42-a156b018d5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:49.251790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925b5c-cb19-4362-af42-a156b018d5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.252075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925b5c-cb19-4362-af42-a156b018d5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.252355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925b5c-cb19-4362-af42-a156b018d5e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.265802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22925b5c-cb19-4362-af42-a156b018d5e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:49.267391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22925b5c-cb19-4362-af42-a156b018d5e0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-22925b5c-cb19-4362-af42-a156b018d5e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:49.287103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:49.289072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:49.291577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:49.291577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:49.291577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:49.291577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:49.295260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 401/781] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:49.297518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22925b5c-cb19-4362-af42-a156b018d5e0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:49.412954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effab27-4e39-4c46-9724-769794bc8ead None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:49.446579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:49.463434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f78321-e71a-43a9-8027-55c5b2e864bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:49.464050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22925b5c-cb19-4362-af42-a156b018d5e0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:49.464170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f78321-e71a-43a9-8027-55c5b2e864bb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.464407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f78321-e71a-43a9-8027-55c5b2e864bb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.464682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f78321-e71a-43a9-8027-55c5b2e864bb None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.465374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f78321-e71a-43a9-8027-55c5b2e864bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:49.466802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f78321-e71a-43a9-8027-55c5b2e864bb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:49.478609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 381/782] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 6 14:08:49 2022] GET /identity/v3/services => generated 772 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:49.492013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9da2e-5786-4b64-813c-ad02c5141da7 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.495250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9da2e-5786-4b64-813c-ad02c5141da7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.496855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9da2e-5786-4b64-813c-ad02c5141da7 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.501869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 382/783] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:49 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:49.525530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e41a-8cc7-45b5-a553-9b0eb554acd0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.526103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e41a-8cc7-45b5-a553-9b0eb554acd0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.526569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e41a-8cc7-45b5-a553-9b0eb554acd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.546965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-25e6e41a-8cc7-45b5-a553-9b0eb554acd0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:49.548500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-25e6e41a-8cc7-45b5-a553-9b0eb554acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:49.554560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174219b-cf42-42c0-b291-00284acad2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:49.555848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174219b-cf42-42c0-b291-00284acad2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.557075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174219b-cf42-42c0-b291-00284acad2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.557075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174219b-cf42-42c0-b291-00284acad2c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.557458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174219b-cf42-42c0-b291-00284acad2c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:49.557971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174219b-cf42-42c0-b291-00284acad2c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:49.559473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174219b-cf42-42c0-b291-00284acad2c0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:49.588997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:49.600424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 402/784] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:49.610857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174219b-cf42-42c0-b291-00284acad2c0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:49.642382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 383/785] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:08:49.666370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6e41a-8cc7-45b5-a553-9b0eb554acd0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:49.783623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6e41a-8cc7-45b5-a553-9b0eb554acd0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:49.820110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174219b-cf42-42c0-b291-00284acad2c0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:49.822116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab4003-4df3-4b9e-88bd-92a67e8f28cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:49.823134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab4003-4df3-4b9e-88bd-92a67e8f28cc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.823604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab4003-4df3-4b9e-88bd-92a67e8f28cc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.824033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab4003-4df3-4b9e-88bd-92a67e8f28cc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.824856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab4003-4df3-4b9e-88bd-92a67e8f28cc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:49.826436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab4003-4df3-4b9e-88bd-92a67e8f28cc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2eab4003-4df3-4b9e-88bd-92a67e8f28cc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:49.842329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:49.843356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 14:08:49.844228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:49.847599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 384/786] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Jun 6 14:08:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:49.854390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab4003-4df3-4b9e-88bd-92a67e8f28cc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:49.864545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda6fd8-5852-4899-a2f6-8f6eaed73848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:49.865466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda6fd8-5852-4899-a2f6-8f6eaed73848 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:49.865849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda6fd8-5852-4899-a2f6-8f6eaed73848 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:49.866237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda6fd8-5852-4899-a2f6-8f6eaed73848 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:49.891532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:49.892755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda6fd8-5852-4899-a2f6-8f6eaed73848 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:49.894741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda6fd8-5852-4899-a2f6-8f6eaed73848 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-cfda6fd8-5852-4899-a2f6-8f6eaed73848 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:49.942631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:49.943637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:49.944578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:49.944578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:49.944578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:49.944578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:49.944578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:49.944578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:49.944578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:49.944578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:49.944578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 403/787] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:49.954892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda6fd8-5852-4899-a2f6-8f6eaed73848 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:49.978460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab4003-4df3-4b9e-88bd-92a67e8f28cc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:49.995996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:50.019624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f37f8-aec7-4662-93c8-637fd3309a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:50.020718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f37f8-aec7-4662-93c8-637fd3309a0c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:50.021379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f37f8-aec7-4662-93c8-637fd3309a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:50.022092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f37f8-aec7-4662-93c8-637fd3309a0c None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:50.023674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f37f8-aec7-4662-93c8-637fd3309a0c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:50.024951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f37f8-aec7-4662-93c8-637fd3309a0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:50.027738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f37f8-aec7-4662-93c8-637fd3309a0c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:50.034717 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 385/788] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 6 14:08:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:50.048580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f37f8-aec7-4662-93c8-637fd3309a0c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:50.170740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda6fd8-5852-4899-a2f6-8f6eaed73848 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:50.244404 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c83b4-4414-4dbe-a56c-c0b77ec366cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:50.245351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c83b4-4414-4dbe-a56c-c0b77ec366cd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:50.245930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c83b4-4414-4dbe-a56c-c0b77ec366cd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:50.246554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c83b4-4414-4dbe-a56c-c0b77ec366cd None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:50.248111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c83b4-4414-4dbe-a56c-c0b77ec366cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:50.248592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c83b4-4414-4dbe-a56c-c0b77ec366cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:50.250890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c83b4-4414-4dbe-a56c-c0b77ec366cd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:50.278103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:50.279309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 404/789] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:50.285422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c83b4-4414-4dbe-a56c-c0b77ec366cd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:50.336720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f37f8-aec7-4662-93c8-637fd3309a0c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:50.420170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ca19c-e9db-4085-bfa7-d208b4327abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:50.420859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ca19c-e9db-4085-bfa7-d208b4327abf None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:50.421140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ca19c-e9db-4085-bfa7-d208b4327abf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:50.421414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ca19c-e9db-4085-bfa7-d208b4327abf None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:50.422170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ca19c-e9db-4085-bfa7-d208b4327abf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:50.422480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ca19c-e9db-4085-bfa7-d208b4327abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:50.423937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ca19c-e9db-4085-bfa7-d208b4327abf None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:50.430967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 386/790] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 6 14:08:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:50.439271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:50.439802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ca19c-e9db-4085-bfa7-d208b4327abf None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:50.483032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c83b4-4414-4dbe-a56c-c0b77ec366cd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:50.542740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb91ceb-c685-495d-8ee7-cf5fce51751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:50.544701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb91ceb-c685-495d-8ee7-cf5fce51751e None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:50.545215 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb91ceb-c685-495d-8ee7-cf5fce51751e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:50.546431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb91ceb-c685-495d-8ee7-cf5fce51751e None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/bb0da8665c1b44f79611129ee1337bdc/roles/4107928942534d3984671dc5232d061d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:50.583142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb91ceb-c685-495d-8ee7-cf5fce51751e None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=bb0da8665c1b44f79611129ee1337bdc, role_id=4107928942534d3984671dc5232d061d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:50.586489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb91ceb-c685-495d-8ee7-cf5fce51751e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:50.628559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 405/791] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:08:50 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/bb0da8665c1b44f79611129ee1337bdc/roles/4107928942534d3984671dc5232d061d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:08:50.661832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ca19c-e9db-4085-bfa7-d208b4327abf None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:50.713427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126e65f-df02-40d3-8cb6-a63eb6b2d790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:50.714157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126e65f-df02-40d3-8cb6-a63eb6b2d790 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:50.714435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126e65f-df02-40d3-8cb6-a63eb6b2d790 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:50.714683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126e65f-df02-40d3-8cb6-a63eb6b2d790 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:50.715505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126e65f-df02-40d3-8cb6-a63eb6b2d790 None admin] RBAC: Adding query filter params (interface=public, service_id=088eb24b40ff43a49d816025cb439bba, region_id=RegionOne) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:50.715808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126e65f-df02-40d3-8cb6-a63eb6b2d790 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:50.717258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126e65f-df02-40d3-8cb6-a63eb6b2d790 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:50.724060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 387/792] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 6 14:08:50 2022] GET /identity/v3/endpoints?service_id=088eb24b40ff43a49d816025cb439bba&interface=public®ion_id=RegionOne => generated 194 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:50.734183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb91ceb-c685-495d-8ee7-cf5fce51751e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:50.751378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151311a0-1c9d-4da0-afa9-5c656f34183f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:50.751669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151311a0-1c9d-4da0-afa9-5c656f34183f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:50.751946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151311a0-1c9d-4da0-afa9-5c656f34183f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:50.753850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 388/793] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:50.761632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5a418-f281-40d5-8e94-f6059730e52b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:50.762080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5a418-f281-40d5-8e94-f6059730e52b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:50.762254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5a418-f281-40d5-8e94-f6059730e52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:50.782294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b4e5a418-f281-40d5-8e94-f6059730e52b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:50.783570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b4e5a418-f281-40d5-8e94-f6059730e52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:50.798861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:50.882686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 389/794] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:08:50.905198 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5a418-f281-40d5-8e94-f6059730e52b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:51.032142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5a418-f281-40d5-8e94-f6059730e52b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:51.050223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb91ceb-c685-495d-8ee7-cf5fce51751e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:51.077527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f580d4-55e2-49ba-9744-f697504fc0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:51.078651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f580d4-55e2-49ba-9744-f697504fc0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:51.078936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f580d4-55e2-49ba-9744-f697504fc0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:51.079218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f580d4-55e2-49ba-9744-f697504fc0c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:51.092500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b8198-f18d-466e-a602-6b551dab4080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:51.093181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b8198-f18d-466e-a602-6b551dab4080 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:51.093461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b8198-f18d-466e-a602-6b551dab4080 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:51.093777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b8198-f18d-466e-a602-6b551dab4080 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:51.094473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f580d4-55e2-49ba-9744-f697504fc0c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:51.094641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b8198-f18d-466e-a602-6b551dab4080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:51.095899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b8198-f18d-466e-a602-6b551dab4080 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:51.098046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:51.103970 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 406/795] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 6 14:08:50 2022] GET /identity/v3/services => generated 772 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-92f580d4-55e2-49ba-9744-f697504fc0c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:51.104238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:51.106053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:51.107744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 390/796] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:51.118135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b8198-f18d-466e-a602-6b551dab4080 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:51.236506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b8198-f18d-466e-a602-6b551dab4080 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:51.285375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1b28f-38ca-4377-8b06-6d3c3cd676a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:51.286169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1b28f-38ca-4377-8b06-6d3c3cd676a0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:51.286451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1b28f-38ca-4377-8b06-6d3c3cd676a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:51.286724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1b28f-38ca-4377-8b06-6d3c3cd676a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:51.287498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1b28f-38ca-4377-8b06-6d3c3cd676a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:51.287801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1b28f-38ca-4377-8b06-6d3c3cd676a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:51.289251 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1b28f-38ca-4377-8b06-6d3c3cd676a0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:51.301057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 407/797] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:51.307739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f580d4-55e2-49ba-9744-f697504fc0c9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:51.318685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:51.457030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f580d4-55e2-49ba-9744-f697504fc0c9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:51.500945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966d61-5a83-4165-95a9-c465ee1d02ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:51.501811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966d61-5a83-4165-95a9-c465ee1d02ca None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:51.505736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966d61-5a83-4165-95a9-c465ee1d02ca None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:51.505736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e966d61-5a83-4165-95a9-c465ee1d02ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:51.520209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e966d61-5a83-4165-95a9-c465ee1d02ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:51.521736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e966d61-5a83-4165-95a9-c465ee1d02ca None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8e966d61-5a83-4165-95a9-c465ee1d02ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:51.543461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:51.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:51.547361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:51.547361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:51.547361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:51.547361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:51.547361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 391/798] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:51.551851 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1b28f-38ca-4377-8b06-6d3c3cd676a0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:51.611495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:51.677062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1b28f-38ca-4377-8b06-6d3c3cd676a0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:51.723110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46237b-fcee-449c-a687-99ba85182047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:51.723800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46237b-fcee-449c-a687-99ba85182047 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:51.724083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46237b-fcee-449c-a687-99ba85182047 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:51.724359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46237b-fcee-449c-a687-99ba85182047 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:51.725101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46237b-fcee-449c-a687-99ba85182047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:51.725408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46237b-fcee-449c-a687-99ba85182047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:51.726847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46237b-fcee-449c-a687-99ba85182047 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:51.741570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 408/799] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:51.751593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966d61-5a83-4165-95a9-c465ee1d02ca None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:51.853019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:51.907576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e966d61-5a83-4165-95a9-c465ee1d02ca None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:51.990325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dbf99-ed56-48aa-a81d-d03e53ca7a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:51.991596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dbf99-ed56-48aa-a81d-d03e53ca7a06 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:51.992090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dbf99-ed56-48aa-a81d-d03e53ca7a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:51.993108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dbf99-ed56-48aa-a81d-d03e53ca7a06 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:52.007278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dbf99-ed56-48aa-a81d-d03e53ca7a06 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:52.010641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dbf99-ed56-48aa-a81d-d03e53ca7a06 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-166dbf99-ed56-48aa-a81d-d03e53ca7a06 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:52.036787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:52.039081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:52.042252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:52.049741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 392/800] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:52.050025 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46237b-fcee-449c-a687-99ba85182047 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:52.204656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46237b-fcee-449c-a687-99ba85182047 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:52.216424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:52.246103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279661e9-aa91-4def-979c-b25a8ccc9d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:52.247700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279661e9-aa91-4def-979c-b25a8ccc9d23 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:52.247988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279661e9-aa91-4def-979c-b25a8ccc9d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:52.248263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279661e9-aa91-4def-979c-b25a8ccc9d23 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:52.249116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279661e9-aa91-4def-979c-b25a8ccc9d23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:52.249424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279661e9-aa91-4def-979c-b25a8ccc9d23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:52.250926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279661e9-aa91-4def-979c-b25a8ccc9d23 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:52.271623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 409/801] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 6 14:08:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:52.283500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dbf99-ed56-48aa-a81d-d03e53ca7a06 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:52.463600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dbf99-ed56-48aa-a81d-d03e53ca7a06 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:52.488565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:52.514917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fc682-a78b-42b2-b95f-3c4c4d751361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:52.515721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc682-a78b-42b2-b95f-3c4c4d751361 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:52.516000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc682-a78b-42b2-b95f-3c4c4d751361 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:52.516276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc682-a78b-42b2-b95f-3c4c4d751361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:52.538478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fc682-a78b-42b2-b95f-3c4c4d751361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:52.540618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fc682-a78b-42b2-b95f-3c4c4d751361 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-209fc682-a78b-42b2-b95f-3c4c4d751361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:52.566100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:52.569182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:52.572544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:52.572544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:52.572544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:52.572544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:52.572544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 393/802] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:52.578865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279661e9-aa91-4def-979c-b25a8ccc9d23 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:52.716383 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45822197-2446-48f1-bdf2-c858c530d600 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:52.716696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45822197-2446-48f1-bdf2-c858c530d600 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:52.716864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45822197-2446-48f1-bdf2-c858c530d600 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:52.722708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 394/803] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:08:52.729345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0b9de-13d9-40fc-8535-7d23e364bbe6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:52.729693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0b9de-13d9-40fc-8535-7d23e364bbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:52.729932 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0b9de-13d9-40fc-8535-7d23e364bbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:52.779741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d7d0b9de-13d9-40fc-8535-7d23e364bbe6 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:52.784731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d7d0b9de-13d9-40fc-8535-7d23e364bbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:52.873118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279661e9-aa91-4def-979c-b25a8ccc9d23 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:52.940454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0adf6-efa5-41da-9092-491c7f4cc72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:52.942095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0adf6-efa5-41da-9092-491c7f4cc72d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:52.942907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0adf6-efa5-41da-9092-491c7f4cc72d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:52.943683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0adf6-efa5-41da-9092-491c7f4cc72d None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:52.945538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0adf6-efa5-41da-9092-491c7f4cc72d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:52.947433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0adf6-efa5-41da-9092-491c7f4cc72d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:52.950421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0adf6-efa5-41da-9092-491c7f4cc72d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:52.973886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 410/804] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:52.977216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec0975-4d06-43bd-913a-0588b6c3d6c5 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:52.978316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec0975-4d06-43bd-913a-0588b6c3d6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:52.979032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec0975-4d06-43bd-913a-0588b6c3d6c5 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:52.983031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 411/805] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:52.984975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec0975-4d06-43bd-913a-0588b6c3d6c5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:53.031314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 395/806] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:08:53.032935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bf34f-a31c-4250-bb7a-762d0cc1e988 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:53.033247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bf34f-a31c-4250-bb7a-762d0cc1e988 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:53.033697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bf34f-a31c-4250-bb7a-762d0cc1e988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:53.055677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:53.087721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5b4bf34f-a31c-4250-bb7a-762d0cc1e988 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:53.090188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5b4bf34f-a31c-4250-bb7a-762d0cc1e988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:53.297977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec0975-4d06-43bd-913a-0588b6c3d6c5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:53.309755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 396/807] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:08:53.309755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bf34f-a31c-4250-bb7a-762d0cc1e988 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:53.365921 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a44f88-adcb-437a-a522-ef22441c3de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:53.367132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a44f88-adcb-437a-a522-ef22441c3de8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:53.367636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a44f88-adcb-437a-a522-ef22441c3de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:53.368093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a44f88-adcb-437a-a522-ef22441c3de8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:53.374583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:53.377538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a44f88-adcb-437a-a522-ef22441c3de8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:53.379516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a44f88-adcb-437a-a522-ef22441c3de8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-60a44f88-adcb-437a-a522-ef22441c3de8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:53.400638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:53.401651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:53.402599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:53.402599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:53.402599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:53.402599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:53.402599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:53.402599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:53.402599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:53.402599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:53.405096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 412/808] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:53.406439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a44f88-adcb-437a-a522-ef22441c3de8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:53.512955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:53.518603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bf34f-a31c-4250-bb7a-762d0cc1e988 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:53.532061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a44f88-adcb-437a-a522-ef22441c3de8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:53.573877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f05d0-7528-4696-8d42-14930f3554b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:53.575075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f05d0-7528-4696-8d42-14930f3554b4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:53.575640 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f05d0-7528-4696-8d42-14930f3554b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:53.576139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f05d0-7528-4696-8d42-14930f3554b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:53.586428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d37a0b-890f-4d5f-b0b6-c04294e0fb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:53.587850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d37a0b-890f-4d5f-b0b6-c04294e0fb5f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:53.588354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d37a0b-890f-4d5f-b0b6-c04294e0fb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:53.588827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d37a0b-890f-4d5f-b0b6-c04294e0fb5f None admin] PATH_INFO: `/v3/services/network` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:53.590093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d37a0b-890f-4d5f-b0b6-c04294e0fb5f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:53.591365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f05d0-7528-4696-8d42-14930f3554b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:53.592639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d37a0b-890f-4d5f-b0b6-c04294e0fb5f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f53f05d0-7528-4696-8d42-14930f3554b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:53.600741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:53.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:53.603305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:53.611375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 413/809] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:53.612706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f05d0-7528-4696-8d42-14930f3554b4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f5d37a0b-890f-4d5f-b0b6-c04294e0fb5f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:53.616867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:53.618967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 14:08:53.619973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:53.622313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 397/810] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:53 2022] GET /identity/v3/services/network => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:53.622313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d37a0b-890f-4d5f-b0b6-c04294e0fb5f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:53.756691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f05d0-7528-4696-8d42-14930f3554b4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:53.774029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:53.787035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d37a0b-890f-4d5f-b0b6-c04294e0fb5f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:53.804557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a882a-6c77-4d8a-afea-a69a92577074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:53.805367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a882a-6c77-4d8a-afea-a69a92577074 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:53.805645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a882a-6c77-4d8a-afea-a69a92577074 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:53.805936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a882a-6c77-4d8a-afea-a69a92577074 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:53.806786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a882a-6c77-4d8a-afea-a69a92577074 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:53.807075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a882a-6c77-4d8a-afea-a69a92577074 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:53.808543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a882a-6c77-4d8a-afea-a69a92577074 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:53.834590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 414/811] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:53.834590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff331216-3d3b-4738-93b8-b1e098185190 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:53.834590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff331216-3d3b-4738-93b8-b1e098185190 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:53.834590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff331216-3d3b-4738-93b8-b1e098185190 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:53.834590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 415/812] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:53.834590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff331216-3d3b-4738-93b8-b1e098185190 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:53.851238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:53.855493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceee991-1b53-4f5c-a277-02bb9c1488ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:53.856337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceee991-1b53-4f5c-a277-02bb9c1488ca None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:53.856700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceee991-1b53-4f5c-a277-02bb9c1488ca None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:53.857114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceee991-1b53-4f5c-a277-02bb9c1488ca None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:53.858269 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceee991-1b53-4f5c-a277-02bb9c1488ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:53.858705 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceee991-1b53-4f5c-a277-02bb9c1488ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:53.861031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceee991-1b53-4f5c-a277-02bb9c1488ca None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:53.877753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 398/813] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:53.879018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceee991-1b53-4f5c-a277-02bb9c1488ca None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:54.022826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff331216-3d3b-4738-93b8-b1e098185190 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:54.060918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceee991-1b53-4f5c-a277-02bb9c1488ca None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:54.078662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e3539-aefb-484d-9df5-c0d07fefc950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:54.079720 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e3539-aefb-484d-9df5-c0d07fefc950 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.080168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e3539-aefb-484d-9df5-c0d07fefc950 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.081011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e3539-aefb-484d-9df5-c0d07fefc950 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.081479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e3539-aefb-484d-9df5-c0d07fefc950 None admin] RBAC: Adding query filter params (name=network) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:54.081871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e3539-aefb-484d-9df5-c0d07fefc950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:54.084199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e3539-aefb-484d-9df5-c0d07fefc950 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:54.093276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 416/814] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:08:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:54.095334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ad9f6-2c94-4463-a8b9-96d8293040b7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.095823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ad9f6-2c94-4463-a8b9-96d8293040b7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.096297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ad9f6-2c94-4463-a8b9-96d8293040b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.125326 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-059ad9f6-2c94-4463-a8b9-96d8293040b7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:54.127128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57abdd-2b81-4f0f-87e6-b5e7b9c0b27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:54.127356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-059ad9f6-2c94-4463-a8b9-96d8293040b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:54.128609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57abdd-2b81-4f0f-87e6-b5e7b9c0b27a None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.129048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57abdd-2b81-4f0f-87e6-b5e7b9c0b27a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.129492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57abdd-2b81-4f0f-87e6-b5e7b9c0b27a None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/3571f3d3a0de4e9abe55ed84426143a0/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.132832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57abdd-2b81-4f0f-87e6-b5e7b9c0b27a None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=3571f3d3a0de4e9abe55ed84426143a0, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:54.135630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57abdd-2b81-4f0f-87e6-b5e7b9c0b27a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:54.165754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 399/815] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:08:53 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/3571f3d3a0de4e9abe55ed84426143a0/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:08:54.165754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57abdd-2b81-4f0f-87e6-b5e7b9c0b27a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:54.198585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:54.257435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:54.266762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 417/816] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:08:54.268896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ad9f6-2c94-4463-a8b9-96d8293040b7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:54.369259 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57abdd-2b81-4f0f-87e6-b5e7b9c0b27a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:54.417439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268ebea-bbac-453d-be0f-91f69f319b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:54.418288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ebea-bbac-453d-be0f-91f69f319b3b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.418568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ebea-bbac-453d-be0f-91f69f319b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.418833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ebea-bbac-453d-be0f-91f69f319b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.432616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268ebea-bbac-453d-be0f-91f69f319b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:54.434255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268ebea-bbac-453d-be0f-91f69f319b3b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:54.447616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ad9f6-2c94-4463-a8b9-96d8293040b7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e268ebea-bbac-453d-be0f-91f69f319b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:54.470484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:54.471580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:54.472652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:54.472652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:54.472652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:54.472652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:54.473492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 400/817] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:54.475330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5b382-277d-45c7-a5f0-eff13b0fe05f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.475788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5b382-277d-45c7-a5f0-eff13b0fe05f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.476138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5b382-277d-45c7-a5f0-eff13b0fe05f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.478449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 401/818] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:54.479602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5b382-277d-45c7-a5f0-eff13b0fe05f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:54.494647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5fa4-2e01-4c94-b987-b5a1ceeff9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:54.495468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5fa4-2e01-4c94-b987-b5a1ceeff9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.495829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5fa4-2e01-4c94-b987-b5a1ceeff9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.496209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5fa4-2e01-4c94-b987-b5a1ceeff9e7 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.496960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f5fa4-2e01-4c94-b987-b5a1ceeff9e7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:54.497274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f5fa4-2e01-4c94-b987-b5a1ceeff9e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:54.498816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f5fa4-2e01-4c94-b987-b5a1ceeff9e7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:54.505260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 418/819] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:08:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:54.511096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5fa4-2e01-4c94-b987-b5a1ceeff9e7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:54.587692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:54.603222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:54.639134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5b382-277d-45c7-a5f0-eff13b0fe05f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:54.656499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5fa4-2e01-4c94-b987-b5a1ceeff9e7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:54.689571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb345f5-3a8f-4a0a-b240-f535000dc2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:54.690478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb345f5-3a8f-4a0a-b240-f535000dc2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.690764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb345f5-3a8f-4a0a-b240-f535000dc2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.691036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb345f5-3a8f-4a0a-b240-f535000dc2b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.706639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb345f5-3a8f-4a0a-b240-f535000dc2b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-6bb345f5-3a8f-4a0a-b240-f535000dc2b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:54.717660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:54.720994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 402/820] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:08:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:54.721916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dd7a2-f2fe-4de3-8ed9-c653d8cf3983 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.722707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dd7a2-f2fe-4de3-8ed9-c653d8cf3983 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.722707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dd7a2-f2fe-4de3-8ed9-c653d8cf3983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.741384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ad975-8bc9-4e2b-9b7f-91c06fa6b925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:54.741805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ad975-8bc9-4e2b-9b7f-91c06fa6b925 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.742116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ad975-8bc9-4e2b-9b7f-91c06fa6b925 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.742482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ad975-8bc9-4e2b-9b7f-91c06fa6b925 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.745227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ad975-8bc9-4e2b-9b7f-91c06fa6b925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:54.745685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ad975-8bc9-4e2b-9b7f-91c06fa6b925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:54.747840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-136dd7a2-f2fe-4de3-8ed9-c653d8cf3983 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:54.748131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ad975-8bc9-4e2b-9b7f-91c06fa6b925 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:54.749817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-136dd7a2-f2fe-4de3-8ed9-c653d8cf3983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:54.774171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 419/821] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:54.774171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ad975-8bc9-4e2b-9b7f-91c06fa6b925 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:54.845431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 403/822] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:08:54.846344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dd7a2-f2fe-4de3-8ed9-c653d8cf3983 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:54.922690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:54.931453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ad975-8bc9-4e2b-9b7f-91c06fa6b925 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:54.957871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:54.971287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10346ab-19fc-449e-8dd1-2fe1c679c257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:54.972056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10346ab-19fc-449e-8dd1-2fe1c679c257 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:54.972434 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10346ab-19fc-449e-8dd1-2fe1c679c257 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:54.972777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10346ab-19fc-449e-8dd1-2fe1c679c257 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:54.973610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10346ab-19fc-449e-8dd1-2fe1c679c257 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:54.975255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10346ab-19fc-449e-8dd1-2fe1c679c257 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:54.979126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dd7a2-f2fe-4de3-8ed9-c653d8cf3983 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:55.000396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 420/823] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 6 14:08:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:08:55.002339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10346ab-19fc-449e-8dd1-2fe1c679c257 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:55.068336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785f8c3-3db9-4db6-a6e0-4f526d1e8fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:55.070208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785f8c3-3db9-4db6-a6e0-4f526d1e8fed None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:55.072658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785f8c3-3db9-4db6-a6e0-4f526d1e8fed None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:55.073505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785f8c3-3db9-4db6-a6e0-4f526d1e8fed None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:55.075359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785f8c3-3db9-4db6-a6e0-4f526d1e8fed None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:55.076303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785f8c3-3db9-4db6-a6e0-4f526d1e8fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:55.079607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785f8c3-3db9-4db6-a6e0-4f526d1e8fed None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:55.099781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 404/824] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:08:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:55.102933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785f8c3-3db9-4db6-a6e0-4f526d1e8fed None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:55.161175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10346ab-19fc-449e-8dd1-2fe1c679c257 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:55.231626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0718-a3e7-4338-9bfe-899cd3579a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:55.232835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0718-a3e7-4338-9bfe-899cd3579a0e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:55.233311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0718-a3e7-4338-9bfe-899cd3579a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:55.233803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0718-a3e7-4338-9bfe-899cd3579a0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:55.253768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e0718-a3e7-4338-9bfe-899cd3579a0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:55.253768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e0718-a3e7-4338-9bfe-899cd3579a0e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d57e0718-a3e7-4338-9bfe-899cd3579a0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:55.280575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:55.282663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:08:55.284826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:55.290412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 421/825] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 6 14:08:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:55.291979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0718-a3e7-4338-9bfe-899cd3579a0e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:55.319884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785f8c3-3db9-4db6-a6e0-4f526d1e8fed None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:55.355208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:55.373876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:55.386136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee9a2a-4f02-4391-a525-ce1ca1222a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:55.387515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee9a2a-4f02-4391-a525-ce1ca1222a59 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:55.387938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee9a2a-4f02-4391-a525-ce1ca1222a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:55.388417 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee9a2a-4f02-4391-a525-ce1ca1222a59 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:55.389080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee9a2a-4f02-4391-a525-ce1ca1222a59 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:55.391649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee9a2a-4f02-4391-a525-ce1ca1222a59 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-13ee9a2a-4f02-4391-a525-ce1ca1222a59 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:55.405350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:55.407117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 14:08:55.408847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:55.410566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 405/826] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Jun 6 14:08:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:55.410566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee9a2a-4f02-4391-a525-ce1ca1222a59 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:55.517790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0718-a3e7-4338-9bfe-899cd3579a0e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:55.566113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e4d7b-497b-4d28-9c8f-b7d1e7fe6048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:55.567535 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e4d7b-497b-4d28-9c8f-b7d1e7fe6048 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:55.568003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e4d7b-497b-4d28-9c8f-b7d1e7fe6048 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:55.568456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e4d7b-497b-4d28-9c8f-b7d1e7fe6048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:55.574533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee9a2a-4f02-4391-a525-ce1ca1222a59 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:55.582759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e4d7b-497b-4d28-9c8f-b7d1e7fe6048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:55.584272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e4d7b-497b-4d28-9c8f-b7d1e7fe6048 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-919e4d7b-497b-4d28-9c8f-b7d1e7fe6048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:55.607868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:55.608861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:55.610056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:55.610056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:55.610056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:55.610056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:55.610056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 422/827] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:55.611505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e4d7b-497b-4d28-9c8f-b7d1e7fe6048 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:55.625178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8366ec9-204b-45d1-b14e-7df65250f9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:55.626314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8366ec9-204b-45d1-b14e-7df65250f9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:55.626640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8366ec9-204b-45d1-b14e-7df65250f9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:55.626901 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8366ec9-204b-45d1-b14e-7df65250f9b6 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:55.627716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8366ec9-204b-45d1-b14e-7df65250f9b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:55.628022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8366ec9-204b-45d1-b14e-7df65250f9b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:55.629482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8366ec9-204b-45d1-b14e-7df65250f9b6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:55.650688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 406/828] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 6 14:08:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:55.652164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8366ec9-204b-45d1-b14e-7df65250f9b6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:55.710134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:55.739546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:55.754206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e4d7b-497b-4d28-9c8f-b7d1e7fe6048 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:55.800749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8366ec9-204b-45d1-b14e-7df65250f9b6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:55.801164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2d595-86b9-4573-989f-a86b729176dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:55.801739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2d595-86b9-4573-989f-a86b729176dd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:55.802095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2d595-86b9-4573-989f-a86b729176dd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:55.802369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2d595-86b9-4573-989f-a86b729176dd None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:55.803089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2d595-86b9-4573-989f-a86b729176dd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:55.803397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2d595-86b9-4573-989f-a86b729176dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:55.804833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2d595-86b9-4573-989f-a86b729176dd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:55.810866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 423/829] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 6 14:08:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:55.815923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2d595-86b9-4573-989f-a86b729176dd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:55.852043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d6d12-99af-4b45-abee-85512e4eec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:55.853263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d6d12-99af-4b45-abee-85512e4eec43 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:55.853692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d6d12-99af-4b45-abee-85512e4eec43 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:55.854535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d6d12-99af-4b45-abee-85512e4eec43 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:55.855416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d6d12-99af-4b45-abee-85512e4eec43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:55.855910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d6d12-99af-4b45-abee-85512e4eec43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:55.857846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d6d12-99af-4b45-abee-85512e4eec43 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:55.878239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 407/830] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:55.880067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d6d12-99af-4b45-abee-85512e4eec43 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:55.952350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2d595-86b9-4573-989f-a86b729176dd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:55.996260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9878e4a-144e-4025-bd0f-6f517bfa8caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:55.997292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878e4a-144e-4025-bd0f-6f517bfa8caa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:55.998033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878e4a-144e-4025-bd0f-6f517bfa8caa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:55.998393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878e4a-144e-4025-bd0f-6f517bfa8caa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:56.013246 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:56.018174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d6d12-99af-4b45-abee-85512e4eec43 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:56.035218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:56.067166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8a343-45f5-46d4-8c1b-bf422c4e53c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:56.068002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8a343-45f5-46d4-8c1b-bf422c4e53c5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:56.068002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8a343-45f5-46d4-8c1b-bf422c4e53c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:56.068276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8a343-45f5-46d4-8c1b-bf422c4e53c5 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:56.068960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8a343-45f5-46d4-8c1b-bf422c4e53c5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:56.069261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8a343-45f5-46d4-8c1b-bf422c4e53c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:56.071298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8a343-45f5-46d4-8c1b-bf422c4e53c5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:56.076639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 408/831] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 6 14:08:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:56.078661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8a343-45f5-46d4-8c1b-bf422c4e53c5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:56.093817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9878e4a-144e-4025-bd0f-6f517bfa8caa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:56.095353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9878e4a-144e-4025-bd0f-6f517bfa8caa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:56.225394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 424/832] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:08:56.227727 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9878e4a-144e-4025-bd0f-6f517bfa8caa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:56.277722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8a343-45f5-46d4-8c1b-bf422c4e53c5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:56.323180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:56.338499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be54c6a-8fe7-4261-a710-e344f6318389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:56.339951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be54c6a-8fe7-4261-a710-e344f6318389 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:56.340454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be54c6a-8fe7-4261-a710-e344f6318389 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:56.340952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be54c6a-8fe7-4261-a710-e344f6318389 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:56.357144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be54c6a-8fe7-4261-a710-e344f6318389 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:56.362328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be54c6a-8fe7-4261-a710-e344f6318389 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7be54c6a-8fe7-4261-a710-e344f6318389 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:56.393502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:56.395636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 409/833] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:08:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:56.397542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be54c6a-8fe7-4261-a710-e344f6318389 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:56.410341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9878e4a-144e-4025-bd0f-6f517bfa8caa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:56.448929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:56.479481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c680e3f-aba9-4f67-bd8d-19632ad6882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:56.481075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c680e3f-aba9-4f67-bd8d-19632ad6882b None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:56.481815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c680e3f-aba9-4f67-bd8d-19632ad6882b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:56.482499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c680e3f-aba9-4f67-bd8d-19632ad6882b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:56.484311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c680e3f-aba9-4f67-bd8d-19632ad6882b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:56.487457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c680e3f-aba9-4f67-bd8d-19632ad6882b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:56.523626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 425/834] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 6 14:08:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:08:56.525651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c680e3f-aba9-4f67-bd8d-19632ad6882b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:56.634793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be54c6a-8fe7-4261-a710-e344f6318389 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:56.725044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c990d31-f2e6-4d32-a2c5-c03b0eafea0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:56.727548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c990d31-f2e6-4d32-a2c5-c03b0eafea0f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:56.728309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c990d31-f2e6-4d32-a2c5-c03b0eafea0f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:56.728996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c990d31-f2e6-4d32-a2c5-c03b0eafea0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:56.754607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c990d31-f2e6-4d32-a2c5-c03b0eafea0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:56.754879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c680e3f-aba9-4f67-bd8d-19632ad6882b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:56.757663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c990d31-f2e6-4d32-a2c5-c03b0eafea0f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:56.783825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8c990d31-f2e6-4d32-a2c5-c03b0eafea0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:56.793415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:56.795091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:56.797067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:56.797067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:56.797067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:56.797067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:56.801930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 410/835] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:56.810290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c990d31-f2e6-4d32-a2c5-c03b0eafea0f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:56.816873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3a1df-83f1-4c66-9539-5065b1d9dc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:56.817624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3a1df-83f1-4c66-9539-5065b1d9dc83 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:56.817948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3a1df-83f1-4c66-9539-5065b1d9dc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:56.818229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3a1df-83f1-4c66-9539-5065b1d9dc83 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:56.819107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3a1df-83f1-4c66-9539-5065b1d9dc83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:56.819422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3a1df-83f1-4c66-9539-5065b1d9dc83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:56.821614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3a1df-83f1-4c66-9539-5065b1d9dc83 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:56.850766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 426/836] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 6 14:08:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:56.861483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3a1df-83f1-4c66-9539-5065b1d9dc83 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:56.955598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:57.004244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c990d31-f2e6-4d32-a2c5-c03b0eafea0f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:57.058128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3a1df-83f1-4c66-9539-5065b1d9dc83 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:57.064773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb4ab0-5592-463f-a7a0-cfe490cf0c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:57.065619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb4ab0-5592-463f-a7a0-cfe490cf0c26 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:57.066024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb4ab0-5592-463f-a7a0-cfe490cf0c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:57.066406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb4ab0-5592-463f-a7a0-cfe490cf0c26 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:57.067264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb4ab0-5592-463f-a7a0-cfe490cf0c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:57.067680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb4ab0-5592-463f-a7a0-cfe490cf0c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:57.069309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb4ab0-5592-463f-a7a0-cfe490cf0c26 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:57.090020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 411/837] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:57.092404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb4ab0-5592-463f-a7a0-cfe490cf0c26 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:57.125636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bd1b7-9d76-474f-8f42-149832e3acea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:57.126534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bd1b7-9d76-474f-8f42-149832e3acea None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:57.126931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bd1b7-9d76-474f-8f42-149832e3acea None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:57.127308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bd1b7-9d76-474f-8f42-149832e3acea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:57.185234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:57.239694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bd1b7-9d76-474f-8f42-149832e3acea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:57.242194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bd1b7-9d76-474f-8f42-149832e3acea None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:57.323324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb4ab0-5592-463f-a7a0-cfe490cf0c26 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:57.377735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:57.389915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2a553-9168-4436-a75b-60821e697772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:57.391197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2a553-9168-4436-a75b-60821e697772 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:57.391641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2a553-9168-4436-a75b-60821e697772 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:57.392037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2a553-9168-4436-a75b-60821e697772 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:57.403428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2a553-9168-4436-a75b-60821e697772 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:57.405618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2a553-9168-4436-a75b-60821e697772 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-b7a2a553-9168-4436-a75b-60821e697772 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:57.430775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:57.431756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:57.432660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:57.432660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:57.432660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:57.432660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:57.432660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:57.432660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:57.432660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:57.432660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:57.433844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 412/838] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:08:57.433971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 427/839] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 14:08:57.435880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528359b2-e60a-4fd2-937b-733f8a431313 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:57.443079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528359b2-e60a-4fd2-937b-733f8a431313 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:57.443338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bd1b7-9d76-474f-8f42-149832e3acea None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:57.443482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528359b2-e60a-4fd2-937b-733f8a431313 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:57.443482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 413/840] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:08:57.444379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528359b2-e60a-4fd2-937b-733f8a431313 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:57.539902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:57.589794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bd1b7-9d76-474f-8f42-149832e3acea None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:57.603505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528359b2-e60a-4fd2-937b-733f8a431313 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:57.647438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aef793-071e-4374-8eb8-a6274822ea6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:57.648122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aef793-071e-4374-8eb8-a6274822ea6a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:57.648410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aef793-071e-4374-8eb8-a6274822ea6a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:57.648752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03062270-b054-42d7-9eb7-0c2b27634c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:57.648964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aef793-071e-4374-8eb8-a6274822ea6a None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:57.649536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aef793-071e-4374-8eb8-a6274822ea6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:57.649698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03062270-b054-42d7-9eb7-0c2b27634c30 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:57.649869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03062270-b054-42d7-9eb7-0c2b27634c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:57.650004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aef793-071e-4374-8eb8-a6274822ea6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:57.650139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03062270-b054-42d7-9eb7-0c2b27634c30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:57.651370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aef793-071e-4374-8eb8-a6274822ea6a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:57.663158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03062270-b054-42d7-9eb7-0c2b27634c30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:57.665155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03062270-b054-42d7-9eb7-0c2b27634c30 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:57.674125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 414/841] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:57.676517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f18c6-b648-4d17-a276-ff8078c463cf None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:57.676810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f18c6-b648-4d17-a276-ff8078c463cf None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:57.677093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f18c6-b648-4d17-a276-ff8078c463cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-03062270-b054-42d7-9eb7-0c2b27634c30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:57.684611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:57.686466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:57.688857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:57.688857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:57.688857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:57.688857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:57.690529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 428/842] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:57.691485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03062270-b054-42d7-9eb7-0c2b27634c30 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:57.695004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-841f18c6-b648-4d17-a276-ff8078c463cf None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:57.696631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-841f18c6-b648-4d17-a276-ff8078c463cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:57.754365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:57.783593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 415/843] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:08:57.785581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f18c6-b648-4d17-a276-ff8078c463cf None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:57.841500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03062270-b054-42d7-9eb7-0c2b27634c30 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:57.878989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:57.893069 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fa5d7-ffd9-48f4-9bd5-d00b8ef8aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:57.893770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fa5d7-ffd9-48f4-9bd5-d00b8ef8aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:57.894062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fa5d7-ffd9-48f4-9bd5-d00b8ef8aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:57.894342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fa5d7-ffd9-48f4-9bd5-d00b8ef8aeb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:57.895195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fa5d7-ffd9-48f4-9bd5-d00b8ef8aeb0 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=0b82a920df4042dd97314350aed4d7a5) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:57.895507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fa5d7-ffd9-48f4-9bd5-d00b8ef8aeb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:57.896977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fa5d7-ffd9-48f4-9bd5-d00b8ef8aeb0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:57.913772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 429/844] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:57 2022] GET /identity/v3/role_assignments?user.id=0b82a920df4042dd97314350aed4d7a5&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:57.913772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fa5d7-ffd9-48f4-9bd5-d00b8ef8aeb0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:57.972535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f18c6-b648-4d17-a276-ff8078c463cf None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:58.041405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fa5d7-ffd9-48f4-9bd5-d00b8ef8aeb0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:58.044323 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102d99b-476c-449c-bdf7-cec92eef0da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:58.046017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102d99b-476c-449c-bdf7-cec92eef0da0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:58.046767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102d99b-476c-449c-bdf7-cec92eef0da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:58.047457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102d99b-476c-449c-bdf7-cec92eef0da0 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:58.049111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102d99b-476c-449c-bdf7-cec92eef0da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:58.049957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102d99b-476c-449c-bdf7-cec92eef0da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:58.052962 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102d99b-476c-449c-bdf7-cec92eef0da0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:58.077454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 416/845] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:58.089743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102d99b-476c-449c-bdf7-cec92eef0da0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:58.090024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c148e-9b7f-4da1-ac4d-fd2a61cac73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:58.090024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c148e-9b7f-4da1-ac4d-fd2a61cac73a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:58.090024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c148e-9b7f-4da1-ac4d-fd2a61cac73a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:58.090024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c148e-9b7f-4da1-ac4d-fd2a61cac73a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:58.101662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c148e-9b7f-4da1-ac4d-fd2a61cac73a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b91c148e-9b7f-4da1-ac4d-fd2a61cac73a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:08:58.118367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:58.120019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:08:58.121264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:58.126174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 430/846] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:58.129091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c148e-9b7f-4da1-ac4d-fd2a61cac73a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:58.143575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:58.205177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:58.245587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102d99b-476c-449c-bdf7-cec92eef0da0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:58.294314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c148e-9b7f-4da1-ac4d-fd2a61cac73a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:58.319119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648abfd2-5bc9-4067-80cd-d00524b38208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:58.320520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648abfd2-5bc9-4067-80cd-d00524b38208 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:58.322061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648abfd2-5bc9-4067-80cd-d00524b38208 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:58.322570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648abfd2-5bc9-4067-80cd-d00524b38208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:58.337545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648abfd2-5bc9-4067-80cd-d00524b38208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:58.339884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3560b-1e59-4498-ad84-8b4269430707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:58.340637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648abfd2-5bc9-4067-80cd-d00524b38208 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:58.340997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3560b-1e59-4498-ad84-8b4269430707 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:58.341462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3560b-1e59-4498-ad84-8b4269430707 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:58.341986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3560b-1e59-4498-ad84-8b4269430707 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:58.342961 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3560b-1e59-4498-ad84-8b4269430707 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:58.343587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3560b-1e59-4498-ad84-8b4269430707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:58.345395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3560b-1e59-4498-ad84-8b4269430707 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:58.356019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 431/847] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:08:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:58.364442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3560b-1e59-4498-ad84-8b4269430707 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-648abfd2-5bc9-4067-80cd-d00524b38208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:08:58.372561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:58.374216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:58.376007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:08:58.376007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:08:58.376007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:08:58.376007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:08:58.376007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:08:58.376007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:08:58.376007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:08:58.376007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:08:58.376007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 417/848] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:08:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:58.385142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648abfd2-5bc9-4067-80cd-d00524b38208 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:58.486311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3560b-1e59-4498-ad84-8b4269430707 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:58.497619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:58.527311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742bd3b-4fe1-4ce4-a6fa-87cf3dd21375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:58.528050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742bd3b-4fe1-4ce4-a6fa-87cf3dd21375 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:58.528338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742bd3b-4fe1-4ce4-a6fa-87cf3dd21375 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:58.528617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742bd3b-4fe1-4ce4-a6fa-87cf3dd21375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:58.540975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742bd3b-4fe1-4ce4-a6fa-87cf3dd21375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:58.542532 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742bd3b-4fe1-4ce4-a6fa-87cf3dd21375 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:58.549212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648abfd2-5bc9-4067-80cd-d00524b38208 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-8742bd3b-4fe1-4ce4-a6fa-87cf3dd21375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:08:58.561909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:58.563027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:58.563931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:08:58.563931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:08:58.563931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:08:58.563931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:58.564158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:58.568028 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 432/849] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:58.569000 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742bd3b-4fe1-4ce4-a6fa-87cf3dd21375 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:58.593877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3718c-b7b4-44b6-8d7b-ac0d9568ecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:58.595011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3718c-b7b4-44b6-8d7b-ac0d9568ecf2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:58.595516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3718c-b7b4-44b6-8d7b-ac0d9568ecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:58.596024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3718c-b7b4-44b6-8d7b-ac0d9568ecf2 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:58.597028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3718c-b7b4-44b6-8d7b-ac0d9568ecf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:58.597538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3718c-b7b4-44b6-8d7b-ac0d9568ecf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:58.599452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3718c-b7b4-44b6-8d7b-ac0d9568ecf2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:58.619748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 418/850] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:58.621171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3718c-b7b4-44b6-8d7b-ac0d9568ecf2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:58.707158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742bd3b-4fe1-4ce4-a6fa-87cf3dd21375 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:58.748231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1df89c-b76f-4120-9e56-1d2c3ba0e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:58.748231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1df89c-b76f-4120-9e56-1d2c3ba0e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:58.748231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1df89c-b76f-4120-9e56-1d2c3ba0e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:58.748747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1df89c-b76f-4120-9e56-1d2c3ba0e5b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:58.749273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1df89c-b76f-4120-9e56-1d2c3ba0e5b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:58.749587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1df89c-b76f-4120-9e56-1d2c3ba0e5b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:58.751059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1df89c-b76f-4120-9e56-1d2c3ba0e5b8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:58.769511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 433/851] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:08:58.770507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1df89c-b76f-4120-9e56-1d2c3ba0e5b8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:58.785917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3718c-b7b4-44b6-8d7b-ac0d9568ecf2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:58.795164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:58.841928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d63e54-4316-43aa-b06b-cfb17adde64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:58.843020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d63e54-4316-43aa-b06b-cfb17adde64e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:58.843600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d63e54-4316-43aa-b06b-cfb17adde64e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:58.844162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d63e54-4316-43aa-b06b-cfb17adde64e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:58.845508 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d63e54-4316-43aa-b06b-cfb17adde64e None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=bb0da8665c1b44f79611129ee1337bdc) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:58.846333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d63e54-4316-43aa-b06b-cfb17adde64e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:58.848273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d63e54-4316-43aa-b06b-cfb17adde64e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:58.866106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 419/852] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:08:58 2022] GET /identity/v3/role_assignments?user.id=bb0da8665c1b44f79611129ee1337bdc&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:58.887695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1df89c-b76f-4120-9e56-1d2c3ba0e5b8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:58.952474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc931c-f5fb-47d1-891b-de9dbc9cab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:58.953937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc931c-f5fb-47d1-891b-de9dbc9cab50 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:58.954387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc931c-f5fb-47d1-891b-de9dbc9cab50 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:58.954885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc931c-f5fb-47d1-891b-de9dbc9cab50 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:58.969770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc931c-f5fb-47d1-891b-de9dbc9cab50 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:58.972448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc931c-f5fb-47d1-891b-de9dbc9cab50 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c8dc931c-f5fb-47d1-891b-de9dbc9cab50 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:08:59.002681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:08:59.004578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:08:59.006501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 434/853] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:08:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:08:59.019591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d63e54-4316-43aa-b06b-cfb17adde64e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:59.088623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:59.195351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d63e54-4316-43aa-b06b-cfb17adde64e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:59.235789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802f830-9392-4cb0-b3c1-53bf60026746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:59.236760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802f830-9392-4cb0-b3c1-53bf60026746 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:59.237199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802f830-9392-4cb0-b3c1-53bf60026746 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:59.237638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802f830-9392-4cb0-b3c1-53bf60026746 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:59.238681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802f830-9392-4cb0-b3c1-53bf60026746 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:08:59.239147 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802f830-9392-4cb0-b3c1-53bf60026746 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:59.240798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802f830-9392-4cb0-b3c1-53bf60026746 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:59.263762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 420/854] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 6 14:08:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:08:59.273313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc931c-f5fb-47d1-891b-de9dbc9cab50 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:59.447719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:59.492301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc931c-f5fb-47d1-891b-de9dbc9cab50 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:08:59.562873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911df69-86a5-4e5f-91a1-9cff963463c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:08:59.564077 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911df69-86a5-4e5f-91a1-9cff963463c2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:59.564567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911df69-86a5-4e5f-91a1-9cff963463c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:59.565113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911df69-86a5-4e5f-91a1-9cff963463c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:59.708420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911df69-86a5-4e5f-91a1-9cff963463c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:08:59.711586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911df69-86a5-4e5f-91a1-9cff963463c2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:08:59.815965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011ab01-571d-4721-a603-f8e3f938d0b8 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:59.816261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011ab01-571d-4721-a603-f8e3f938d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:59.816605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011ab01-571d-4721-a603-f8e3f938d0b8 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:59.823058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 421/855] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:08:59.825103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2c09c-62bd-4158-b462-2a1a694b6790 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:08:59.825397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2c09c-62bd-4158-b462-2a1a694b6790 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:08:59.825655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2c09c-62bd-4158-b462-2a1a694b6790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:08:59.844949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5da2c09c-62bd-4158-b462-2a1a694b6790 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:08:59.846296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5da2c09c-62bd-4158-b462-2a1a694b6790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:08:59.875241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:08:59.917870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 435/856] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:08:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:08:59.925044 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911df69-86a5-4e5f-91a1-9cff963463c2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:08:59.952037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 422/857] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:08:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:08:59.989369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2c09c-62bd-4158-b462-2a1a694b6790 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:00.012693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:00.101692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911df69-86a5-4e5f-91a1-9cff963463c2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:00.168599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2182d76-c580-4021-be7d-d783bdde02f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:00.170090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2182d76-c580-4021-be7d-d783bdde02f5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:00.170090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2182d76-c580-4021-be7d-d783bdde02f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:00.170396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2182d76-c580-4021-be7d-d783bdde02f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:00.188783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2182d76-c580-4021-be7d-d783bdde02f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:00.190812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2182d76-c580-4021-be7d-d783bdde02f5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:00.214204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2c09c-62bd-4158-b462-2a1a694b6790 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d2182d76-c580-4021-be7d-d783bdde02f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:00.218850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:00.220543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:00.222302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:00.222302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:00.222302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:00.222302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:00.226053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 436/858] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:00.228560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2182d76-c580-4021-be7d-d783bdde02f5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:00.278266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488a796-d7d2-4c3d-9e7e-9ff490267500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:00.279517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488a796-d7d2-4c3d-9e7e-9ff490267500 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:00.279517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488a796-d7d2-4c3d-9e7e-9ff490267500 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:00.279835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488a796-d7d2-4c3d-9e7e-9ff490267500 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:00.280280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488a796-d7d2-4c3d-9e7e-9ff490267500 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:00.281691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488a796-d7d2-4c3d-9e7e-9ff490267500 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c488a796-d7d2-4c3d-9e7e-9ff490267500 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:00.295922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:00.297906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 14:09:00.299576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:00.302573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 423/859] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:08:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:00.303374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488a796-d7d2-4c3d-9e7e-9ff490267500 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:00.311428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:00.411047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:00.417955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2182d76-c580-4021-be7d-d783bdde02f5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:00.441376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488a796-d7d2-4c3d-9e7e-9ff490267500 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:00.471240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed127d-ce36-4557-8628-32ae7b828874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:00.472282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed127d-ce36-4557-8628-32ae7b828874 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:00.472593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed127d-ce36-4557-8628-32ae7b828874 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:00.472926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed127d-ce36-4557-8628-32ae7b828874 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:00.474371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed127d-ce36-4557-8628-32ae7b828874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:00.474757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed127d-ce36-4557-8628-32ae7b828874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:00.478224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed127d-ce36-4557-8628-32ae7b828874 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:00.482311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642454b-8dc2-4891-af87-6055c43d160f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:00.483288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642454b-8dc2-4891-af87-6055c43d160f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:00.483777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642454b-8dc2-4891-af87-6055c43d160f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:00.484256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642454b-8dc2-4891-af87-6055c43d160f None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:00.485432 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642454b-8dc2-4891-af87-6055c43d160f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:00.486009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642454b-8dc2-4891-af87-6055c43d160f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:00.488785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642454b-8dc2-4891-af87-6055c43d160f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:00.495627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 424/860] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:09:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:00.504244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642454b-8dc2-4891-af87-6055c43d160f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:00.513501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 437/861] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:00.516992 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed127d-ce36-4557-8628-32ae7b828874 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:00.645151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:00.655824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed127d-ce36-4557-8628-32ae7b828874 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:00.695167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac675619-f4d8-4fac-8011-b0cd1b406790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:00.703198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac675619-f4d8-4fac-8011-b0cd1b406790 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:00.703762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac675619-f4d8-4fac-8011-b0cd1b406790 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:00.704378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac675619-f4d8-4fac-8011-b0cd1b406790 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:00.718085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac675619-f4d8-4fac-8011-b0cd1b406790 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:00.719176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642454b-8dc2-4891-af87-6055c43d160f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:00.721156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac675619-f4d8-4fac-8011-b0cd1b406790 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-ac675619-f4d8-4fac-8011-b0cd1b406790 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:09:00.749792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:00.750789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:00.751707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:09:00.751707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:09:00.751707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:09:00.751707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:09:00.751707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:09:00.751707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:09:00.751707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:00.751707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:00.751707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 438/862] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:00.755695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c23e-2339-434a-8ae2-84b356c8eba0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:00.755695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c23e-2339-434a-8ae2-84b356c8eba0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:00.755695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c23e-2339-434a-8ae2-84b356c8eba0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:00.760279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 439/863] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:00.763366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c23e-2339-434a-8ae2-84b356c8eba0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:00.793897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e77f49-2e51-45ca-8e5b-c81562265a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:00.795035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e77f49-2e51-45ca-8e5b-c81562265a10 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:00.795542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e77f49-2e51-45ca-8e5b-c81562265a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:00.796043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e77f49-2e51-45ca-8e5b-c81562265a10 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:00.797356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e77f49-2e51-45ca-8e5b-c81562265a10 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:00.797942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e77f49-2e51-45ca-8e5b-c81562265a10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:00.800468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e77f49-2e51-45ca-8e5b-c81562265a10 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:00.809509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 425/864] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:09:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:00.811390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320500b-6552-4410-b983-d132dac6f4b2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:00.811864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320500b-6552-4410-b983-d132dac6f4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:00.812346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320500b-6552-4410-b983-d132dac6f4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:00.824894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:00.846059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3320500b-6552-4410-b983-d132dac6f4b2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:00.848286 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3320500b-6552-4410-b983-d132dac6f4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:00.980413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 426/865] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:00.981351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c23e-2339-434a-8ae2-84b356c8eba0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:01.003316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320500b-6552-4410-b983-d132dac6f4b2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:01.048854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3920bfe-f86f-4d94-af58-1b4c583e4c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:01.049538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3920bfe-f86f-4d94-af58-1b4c583e4c85 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:01.049878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3920bfe-f86f-4d94-af58-1b4c583e4c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:01.050165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3920bfe-f86f-4d94-af58-1b4c583e4c85 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:01.050974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3920bfe-f86f-4d94-af58-1b4c583e4c85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:01.051288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3920bfe-f86f-4d94-af58-1b4c583e4c85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:01.052884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3920bfe-f86f-4d94-af58-1b4c583e4c85 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:01.066792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 440/866] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:09:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:01.074821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3920bfe-f86f-4d94-af58-1b4c583e4c85 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:01.098718 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:01.125559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320500b-6552-4410-b983-d132dac6f4b2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:01.167066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d807e6-f2cc-403b-823c-5661fbd682cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:01.167886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d807e6-f2cc-403b-823c-5661fbd682cf None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:01.168160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d807e6-f2cc-403b-823c-5661fbd682cf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:01.168427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d807e6-f2cc-403b-823c-5661fbd682cf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:01.169083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d807e6-f2cc-403b-823c-5661fbd682cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:01.170528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d807e6-f2cc-403b-823c-5661fbd682cf None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:01.172621 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-95d807e6-f2cc-403b-823c-5661fbd682cf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:01.186113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:01.188243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 14:09:01.190292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:01.192479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 427/867] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Jun 6 14:09:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:01.193425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d807e6-f2cc-403b-823c-5661fbd682cf None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:01.270593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3920bfe-f86f-4d94-af58-1b4c583e4c85 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:01.310002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d807e6-f2cc-403b-823c-5661fbd682cf None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:01.316750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9350c-a98e-4c44-9bc3-11cac309d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:01.317987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9350c-a98e-4c44-9bc3-11cac309d2da None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:01.318501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9350c-a98e-4c44-9bc3-11cac309d2da None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:01.318977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9350c-a98e-4c44-9bc3-11cac309d2da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:01.320139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9350c-a98e-4c44-9bc3-11cac309d2da None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=3571f3d3a0de4e9abe55ed84426143a0) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:01.321206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9350c-a98e-4c44-9bc3-11cac309d2da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:01.323280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9350c-a98e-4c44-9bc3-11cac309d2da None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:01.330837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 441/868] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:09:01 2022] GET /identity/v3/role_assignments?user.id=3571f3d3a0de4e9abe55ed84426143a0&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:01.354239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839a02e-aba6-40b4-9485-f16c04ede396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:01.355400 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839a02e-aba6-40b4-9485-f16c04ede396 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:01.355894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839a02e-aba6-40b4-9485-f16c04ede396 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:01.356375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839a02e-aba6-40b4-9485-f16c04ede396 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:01.357398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839a02e-aba6-40b4-9485-f16c04ede396 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:01.358000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839a02e-aba6-40b4-9485-f16c04ede396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:01.359932 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839a02e-aba6-40b4-9485-f16c04ede396 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:01.366462 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 428/869] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:01.374948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9350c-a98e-4c44-9bc3-11cac309d2da None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:01.490346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:01.517573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9350c-a98e-4c44-9bc3-11cac309d2da None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:01.565674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4fe6f-bac9-4028-8689-5b73c039d2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:01.566403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4fe6f-bac9-4028-8689-5b73c039d2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:01.566688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4fe6f-bac9-4028-8689-5b73c039d2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:01.566961 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4fe6f-bac9-4028-8689-5b73c039d2a4 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:01.567695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4fe6f-bac9-4028-8689-5b73c039d2a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:01.568059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4fe6f-bac9-4028-8689-5b73c039d2a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:01.569439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4fe6f-bac9-4028-8689-5b73c039d2a4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:01.575641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 442/870] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:01.826659 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14b0ee-5c2f-4af5-80de-3ecb797c1c85 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:01.827080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14b0ee-5c2f-4af5-80de-3ecb797c1c85 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:01.827562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14b0ee-5c2f-4af5-80de-3ecb797c1c85 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:01.830583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 429/871] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:01.841073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29bc2-850d-4b40-9f8f-57c01a882072 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:01.841893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29bc2-850d-4b40-9f8f-57c01a882072 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:01.842171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29bc2-850d-4b40-9f8f-57c01a882072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:01.870572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-04d29bc2-850d-4b40-9f8f-57c01a882072 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:01.872455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-04d29bc2-850d-4b40-9f8f-57c01a882072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:01.957742 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 443/872] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:01.982805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14b0ee-5c2f-4af5-80de-3ecb797c1c85 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:02.065283 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:02.111399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14b0ee-5c2f-4af5-80de-3ecb797c1c85 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:02.149766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0e765-25a7-4105-b8b7-9315e2618fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:02.150709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e765-25a7-4105-b8b7-9315e2618fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:02.150993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e765-25a7-4105-b8b7-9315e2618fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:02.151269 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e765-25a7-4105-b8b7-9315e2618fb8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:02.163030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0e765-25a7-4105-b8b7-9315e2618fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d6c0e765-25a7-4105-b8b7-9315e2618fb8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:09:02.172150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:02.173193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:02.174082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 430/873] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 6 14:09:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:02.184185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d29bc2-850d-4b40-9f8f-57c01a882072 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:02.284041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:02.319475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d29bc2-850d-4b40-9f8f-57c01a882072 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:02.369656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf7287-1aba-4e32-af47-df4ec91f5540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:02.370350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf7287-1aba-4e32-af47-df4ec91f5540 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:02.370640 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf7287-1aba-4e32-af47-df4ec91f5540 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:02.371062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf7287-1aba-4e32-af47-df4ec91f5540 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:02.372030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf7287-1aba-4e32-af47-df4ec91f5540 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:02.372360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf7287-1aba-4e32-af47-df4ec91f5540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:02.374032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf7287-1aba-4e32-af47-df4ec91f5540 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:02.390294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 444/874] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 6 14:09:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:02.395699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0e765-25a7-4105-b8b7-9315e2618fb8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:02.532444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0e765-25a7-4105-b8b7-9315e2618fb8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:02.562132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:02.570104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd66ef7-5928-491f-b9b0-66570bda2f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:02.571442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd66ef7-5928-491f-b9b0-66570bda2f0e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:02.571787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd66ef7-5928-491f-b9b0-66570bda2f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:02.572059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd66ef7-5928-491f-b9b0-66570bda2f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:02.584128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd66ef7-5928-491f-b9b0-66570bda2f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:02.585586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd66ef7-5928-491f-b9b0-66570bda2f0e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-dbd66ef7-5928-491f-b9b0-66570bda2f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:02.606261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:02.607232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:02.608826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:02.608826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:02.608826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:02.608826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:02.608826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 431/875] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:02.613565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf7287-1aba-4e32-af47-df4ec91f5540 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:02.767486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf7287-1aba-4e32-af47-df4ec91f5540 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:02.829473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:02.840423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee903bd-6c1a-4b93-b79c-d610b9274143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:02.841618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee903bd-6c1a-4b93-b79c-d610b9274143 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:02.842281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee903bd-6c1a-4b93-b79c-d610b9274143 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:02.842669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee903bd-6c1a-4b93-b79c-d610b9274143 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:02.844081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee903bd-6c1a-4b93-b79c-d610b9274143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:02.844656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee903bd-6c1a-4b93-b79c-d610b9274143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:02.847379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee903bd-6c1a-4b93-b79c-d610b9274143 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:02.870265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 445/876] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:02.878492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd66ef7-5928-491f-b9b0-66570bda2f0e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:03.000639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd66ef7-5928-491f-b9b0-66570bda2f0e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:03.046105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a4199-6d8f-488e-ab9c-4c2301f13348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:03.047431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a4199-6d8f-488e-ab9c-4c2301f13348 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:03.047924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a4199-6d8f-488e-ab9c-4c2301f13348 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:03.048420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a4199-6d8f-488e-ab9c-4c2301f13348 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:03.057968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a4199-6d8f-488e-ab9c-4c2301f13348 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:03.059775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a4199-6d8f-488e-ab9c-4c2301f13348 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a20a4199-6d8f-488e-ab9c-4c2301f13348 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:09:03.075943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:03.077036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:09:03.078088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:03.080907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 432/877] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:09:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:03.088052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee903bd-6c1a-4b93-b79c-d610b9274143 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:03.214609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee903bd-6c1a-4b93-b79c-d610b9274143 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:03.252756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17840867-63db-4018-9c2d-d3980f05e2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:03.253499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17840867-63db-4018-9c2d-d3980f05e2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:03.253817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17840867-63db-4018-9c2d-d3980f05e2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:03.254097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17840867-63db-4018-9c2d-d3980f05e2d4 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:03.254954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17840867-63db-4018-9c2d-d3980f05e2d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:03.255269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17840867-63db-4018-9c2d-d3980f05e2d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:03.256742 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17840867-63db-4018-9c2d-d3980f05e2d4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:03.278126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 446/878] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 6 14:09:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:03.283600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a4199-6d8f-488e-ab9c-4c2301f13348 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:03.322804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:03.425020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a4199-6d8f-488e-ab9c-4c2301f13348 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:03.480620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5831097-3a48-42c5-802d-a8edc41102b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:03.481450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5831097-3a48-42c5-802d-a8edc41102b8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:03.481780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5831097-3a48-42c5-802d-a8edc41102b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:03.482201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5831097-3a48-42c5-802d-a8edc41102b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:03.602270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5831097-3a48-42c5-802d-a8edc41102b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:03.606141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5831097-3a48-42c5-802d-a8edc41102b8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:03.616659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea021c-856c-4943-8300-9a3e90735153 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:03.617689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea021c-856c-4943-8300-9a3e90735153 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:03.618539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea021c-856c-4943-8300-9a3e90735153 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:03.628073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 447/879] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:03.631663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb419f-ad95-4323-ac67-5f6101a71ada None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:03.632562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb419f-ad95-4323-ac67-5f6101a71ada None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:03.633945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb419f-ad95-4323-ac67-5f6101a71ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:03.661906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d9bb419f-ad95-4323-ac67-5f6101a71ada None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:03.666395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d9bb419f-ad95-4323-ac67-5f6101a71ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:03.678023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:03.778998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:03.812428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 433/880] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:09:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:09:03.821099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5831097-3a48-42c5-802d-a8edc41102b8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:03.825587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 448/881] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:03.856886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb419f-ad95-4323-ac67-5f6101a71ada None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:03.940460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5831097-3a48-42c5-802d-a8edc41102b8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:03.980846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce3d2d-a7f4-435c-8a0b-bf6172e31759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:03.982104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce3d2d-a7f4-435c-8a0b-bf6172e31759 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:03.982602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce3d2d-a7f4-435c-8a0b-bf6172e31759 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:03.983087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce3d2d-a7f4-435c-8a0b-bf6172e31759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:03.986006 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:03.997007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce3d2d-a7f4-435c-8a0b-bf6172e31759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:03.998901 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce3d2d-a7f4-435c-8a0b-bf6172e31759 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d3ce3d2d-a7f4-435c-8a0b-bf6172e31759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:04.019477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:04.020561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:04.021470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:04.021470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:04.021470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:04.021470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:04.024039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 434/882] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:04.029600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce3d2d-a7f4-435c-8a0b-bf6172e31759 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:04.036310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb419f-ad95-4323-ac67-5f6101a71ada None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:04.084369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7dbd1-7362-4f66-8879-dabeb857e536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:04.085414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7dbd1-7362-4f66-8879-dabeb857e536 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:04.085916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7dbd1-7362-4f66-8879-dabeb857e536 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:04.086455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7dbd1-7362-4f66-8879-dabeb857e536 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:04.087307 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7dbd1-7362-4f66-8879-dabeb857e536 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:04.088868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7dbd1-7362-4f66-8879-dabeb857e536 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:04.113850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 449/883] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:09:03 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:09:04.113850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a199a5-010c-4f72-b82d-b1c21e27fd2e None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:04.113850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a199a5-010c-4f72-b82d-b1c21e27fd2e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:04.113850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a199a5-010c-4f72-b82d-b1c21e27fd2e None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:04.113850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 450/884] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:04.122413 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44fab5-3a13-4526-8a6d-37ab7a797df3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:04.122690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44fab5-3a13-4526-8a6d-37ab7a797df3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:04.122968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44fab5-3a13-4526-8a6d-37ab7a797df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:04.143242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ff44fab5-3a13-4526-8a6d-37ab7a797df3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:04.144636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ff44fab5-3a13-4526-8a6d-37ab7a797df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:04.191138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce3d2d-a7f4-435c-8a0b-bf6172e31759 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:04.216450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:04.249300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6869b-a0f2-49aa-a5f5-8157042d77f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:04.250078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6869b-a0f2-49aa-a5f5-8157042d77f7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:04.250401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6869b-a0f2-49aa-a5f5-8157042d77f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:04.250678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6869b-a0f2-49aa-a5f5-8157042d77f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:04.251440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6869b-a0f2-49aa-a5f5-8157042d77f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:04.251741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6869b-a0f2-49aa-a5f5-8157042d77f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:04.253155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6869b-a0f2-49aa-a5f5-8157042d77f7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:04.268329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 435/885] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:04.275989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6869b-a0f2-49aa-a5f5-8157042d77f7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:04.300008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 451/886] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:04.323906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44fab5-3a13-4526-8a6d-37ab7a797df3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:04.347524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:04.443913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6869b-a0f2-49aa-a5f5-8157042d77f7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:04.501818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff2f95-04a1-4428-a5c8-73c02713b67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:04.502724 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff2f95-04a1-4428-a5c8-73c02713b67f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:04.503004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff2f95-04a1-4428-a5c8-73c02713b67f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:04.503280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff2f95-04a1-4428-a5c8-73c02713b67f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:04.513573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff2f95-04a1-4428-a5c8-73c02713b67f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:04.515793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff2f95-04a1-4428-a5c8-73c02713b67f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:04.536412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44fab5-3a13-4526-8a6d-37ab7a797df3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9fff2f95-04a1-4428-a5c8-73c02713b67f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:09:04.541732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:04.543311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:04.545645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:09:04.545645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:09:04.545645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:09:04.545645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:09:04.545645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:09:04.545645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:09:04.545645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:04.545645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:04.545645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 436/887] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:04.546555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3127a74-615b-41ea-aa95-4f54b0010c55 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:04.546555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3127a74-615b-41ea-aa95-4f54b0010c55 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:04.549777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3127a74-615b-41ea-aa95-4f54b0010c55 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:04.550160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 437/888] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:04.559693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c714b1-1850-4599-b5ca-b62a6672c60f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:04.559693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c714b1-1850-4599-b5ca-b62a6672c60f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:04.559693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c714b1-1850-4599-b5ca-b62a6672c60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:04.587576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-71c714b1-1850-4599-b5ca-b62a6672c60f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:04.589187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-71c714b1-1850-4599-b5ca-b62a6672c60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:04.605900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddcdbc-5eae-453c-b537-9e108b95bcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:04.607124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddcdbc-5eae-453c-b537-9e108b95bcc7 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:04.607581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddcdbc-5eae-453c-b537-9e108b95bcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:04.608037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddcdbc-5eae-453c-b537-9e108b95bcc7 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:04.609006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddcdbc-5eae-453c-b537-9e108b95bcc7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:04.611734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddcdbc-5eae-453c-b537-9e108b95bcc7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:04.634670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 452/889] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:09:04 2022] POST /identity/v3/services => generated 269 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:09:04.636024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddcdbc-5eae-453c-b537-9e108b95bcc7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:04.703848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:04.791298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 438/890] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:04.832652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c714b1-1850-4599-b5ca-b62a6672c60f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:04.878456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddcdbc-5eae-453c-b537-9e108b95bcc7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:04.893625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:04.950960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9eb85-48c2-4df9-9d34-56bdb18c42f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:04.952395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9eb85-48c2-4df9-9d34-56bdb18c42f7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:04.953080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9eb85-48c2-4df9-9d34-56bdb18c42f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:04.953772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9eb85-48c2-4df9-9d34-56bdb18c42f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:04.955482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9eb85-48c2-4df9-9d34-56bdb18c42f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:04.956224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9eb85-48c2-4df9-9d34-56bdb18c42f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:04.959297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9eb85-48c2-4df9-9d34-56bdb18c42f7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:04.989901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 453/891] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:09:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:04.993942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9eb85-48c2-4df9-9d34-56bdb18c42f7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:05.032386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c714b1-1850-4599-b5ca-b62a6672c60f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:05.100629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368714fa-388f-4468-9328-037b1626987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:05.102788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368714fa-388f-4468-9328-037b1626987b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:05.104018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368714fa-388f-4468-9328-037b1626987b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:05.104449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368714fa-388f-4468-9328-037b1626987b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:05.105670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368714fa-388f-4468-9328-037b1626987b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:05.108736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368714fa-388f-4468-9328-037b1626987b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-368714fa-388f-4468-9328-037b1626987b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:05.133657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:05.135935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:05.138110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 439/892] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:05.146210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368714fa-388f-4468-9328-037b1626987b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:05.190187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:05.195154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9eb85-48c2-4df9-9d34-56bdb18c42f7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:05.264674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ce599-4ed0-49e3-b912-966128cce22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:05.266302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ce599-4ed0-49e3-b912-966128cce22b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:05.267113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ce599-4ed0-49e3-b912-966128cce22b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:05.267858 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ce599-4ed0-49e3-b912-966128cce22b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:05.269956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ce599-4ed0-49e3-b912-966128cce22b None admin] RBAC: Adding query filter params (role.id=e57912abab304c40b99b65e14077a804, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=bb0da8665c1b44f79611129ee1337bdc) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:05.270819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ce599-4ed0-49e3-b912-966128cce22b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:05.273976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ce599-4ed0-49e3-b912-966128cce22b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:05.294371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 454/893] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:09:04 2022] GET /identity/v3/role_assignments?user.id=bb0da8665c1b44f79611129ee1337bdc&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=e57912abab304c40b99b65e14077a804 => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:05.345356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:05.369339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368714fa-388f-4468-9328-037b1626987b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:05.447635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2d98-9c83-4fc6-ad7b-8bcf4e0d94e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:05.448818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2d98-9c83-4fc6-ad7b-8bcf4e0d94e4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:05.449321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2d98-9c83-4fc6-ad7b-8bcf4e0d94e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:05.449837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2d98-9c83-4fc6-ad7b-8bcf4e0d94e4 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:05.451180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2d98-9c83-4fc6-ad7b-8bcf4e0d94e4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:05.451696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2d98-9c83-4fc6-ad7b-8bcf4e0d94e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:05.454333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2d98-9c83-4fc6-ad7b-8bcf4e0d94e4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:05.465636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 440/894] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:05.475059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ce599-4ed0-49e3-b912-966128cce22b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:05.707568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ce599-4ed0-49e3-b912-966128cce22b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:05.772159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9396d-9941-4f99-917c-5a29f16d75f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:05.773799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9396d-9941-4f99-917c-5a29f16d75f8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:05.774588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9396d-9941-4f99-917c-5a29f16d75f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:05.775363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9396d-9941-4f99-917c-5a29f16d75f8 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:05.776985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9396d-9941-4f99-917c-5a29f16d75f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:05.777824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9396d-9941-4f99-917c-5a29f16d75f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:05.780373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9396d-9941-4f99-917c-5a29f16d75f8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:05.789940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 455/895] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:07.114394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bfa33-4d82-42e8-a20a-b5e8386d8ee5 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:07.114960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bfa33-4d82-42e8-a20a-b5e8386d8ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:07.114960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bfa33-4d82-42e8-a20a-b5e8386d8ee5 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:07.116672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 441/896] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:07.122843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7052c-ebf7-491c-b3de-a617e2c3dc3b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:07.123408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7052c-ebf7-491c-b3de-a617e2c3dc3b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:07.123408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7052c-ebf7-491c-b3de-a617e2c3dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:07.142219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f1e7052c-ebf7-491c-b3de-a617e2c3dc3b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:07.143466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f1e7052c-ebf7-491c-b3de-a617e2c3dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:07.158964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:07.221895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 456/897] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:07.250121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bfa33-4d82-42e8-a20a-b5e8386d8ee5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:07.431142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bfa33-4d82-42e8-a20a-b5e8386d8ee5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:07.498153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93e196-92d8-4d29-80ea-7ba01fd1d649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:07.499511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93e196-92d8-4d29-80ea-7ba01fd1d649 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:07.499999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93e196-92d8-4d29-80ea-7ba01fd1d649 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:07.500486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93e196-92d8-4d29-80ea-7ba01fd1d649 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:07.505588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93e196-92d8-4d29-80ea-7ba01fd1d649 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:07.505588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93e196-92d8-4d29-80ea-7ba01fd1d649 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-eb93e196-92d8-4d29-80ea-7ba01fd1d649 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:07.529955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:07.531936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:07.534501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 442/898] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:09:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:07.542107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7052c-ebf7-491c-b3de-a617e2c3dc3b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:07.687865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7052c-ebf7-491c-b3de-a617e2c3dc3b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:07.727108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c8d47-a6e7-444d-8e0c-467308edd07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:07.727746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c8d47-a6e7-444d-8e0c-467308edd07d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:07.728008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c8d47-a6e7-444d-8e0c-467308edd07d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:07.728252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c8d47-a6e7-444d-8e0c-467308edd07d None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:07.728818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c8d47-a6e7-444d-8e0c-467308edd07d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:07.730891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c8d47-a6e7-444d-8e0c-467308edd07d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:07.730891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c8d47-a6e7-444d-8e0c-467308edd07d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:07.736683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 457/899] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:09:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:07.746142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93e196-92d8-4d29-80ea-7ba01fd1d649 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:07.788296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:07.890008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963a2d1-c9a7-4fa4-8121-a3c4f2146f89 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:07.890493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963a2d1-c9a7-4fa4-8121-a3c4f2146f89 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:07.890971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963a2d1-c9a7-4fa4-8121-a3c4f2146f89 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:07.899507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 458/900] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:07.903643 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dad4e-fa63-4d74-979b-87508cc82146 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:07.904155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dad4e-fa63-4d74-979b-87508cc82146 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:07.904633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dad4e-fa63-4d74-979b-87508cc82146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:07.904889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93e196-92d8-4d29-80ea-7ba01fd1d649 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:07.934770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6b1dad4e-fa63-4d74-979b-87508cc82146 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:07.936918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6b1dad4e-fa63-4d74-979b-87508cc82146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:07.947184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a93f8c-38d4-4521-a6a6-b86f2b6727e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:07.947765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a93f8c-38d4-4521-a6a6-b86f2b6727e6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:07.948034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a93f8c-38d4-4521-a6a6-b86f2b6727e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:07.948310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a93f8c-38d4-4521-a6a6-b86f2b6727e6 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:07.949035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a93f8c-38d4-4521-a6a6-b86f2b6727e6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:07.949342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a93f8c-38d4-4521-a6a6-b86f2b6727e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:07.950827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a93f8c-38d4-4521-a6a6-b86f2b6727e6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:07.952703 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:07.960799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 443/901] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:09:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:07.968836 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a93f8c-38d4-4521-a6a6-b86f2b6727e6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:08.086895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 459/902] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:08.123968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dad4e-fa63-4d74-979b-87508cc82146 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:08.138352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:08.150704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a93f8c-38d4-4521-a6a6-b86f2b6727e6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:08.192206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7a7e2-5720-4ee9-af40-3220a6ea6e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:08.192902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7a7e2-5720-4ee9-af40-3220a6ea6e0b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:08.193195 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7a7e2-5720-4ee9-af40-3220a6ea6e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:08.193477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7a7e2-5720-4ee9-af40-3220a6ea6e0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:08.194377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7a7e2-5720-4ee9-af40-3220a6ea6e0b None admin] RBAC: Adding query filter params (interface=public, service_id=fc439974d8934bdfa75b9054cb888c29, region_id=RegionOne) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:08.194696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7a7e2-5720-4ee9-af40-3220a6ea6e0b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:08.196413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7a7e2-5720-4ee9-af40-3220a6ea6e0b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:08.203230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 444/903] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 6 14:09:07 2022] GET /identity/v3/endpoints?service_id=fc439974d8934bdfa75b9054cb888c29&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:08.215663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7a7e2-5720-4ee9-af40-3220a6ea6e0b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:08.294106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dad4e-fa63-4d74-979b-87508cc82146 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:08.364076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d92a8-1002-407b-83e5-f6ec2850a26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:08.365966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d92a8-1002-407b-83e5-f6ec2850a26c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:08.366765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d92a8-1002-407b-83e5-f6ec2850a26c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:08.368755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d92a8-1002-407b-83e5-f6ec2850a26c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:08.370437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d92a8-1002-407b-83e5-f6ec2850a26c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:08.373468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7a7e2-5720-4ee9-af40-3220a6ea6e0b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:08.373803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d92a8-1002-407b-83e5-f6ec2850a26c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-952d92a8-1002-407b-83e5-f6ec2850a26c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:08.398348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:08.400730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 14:09:08.402364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:08.406746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 460/904] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Jun 6 14:09:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:08.413181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d92a8-1002-407b-83e5-f6ec2850a26c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:08.425460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9adfae-1af4-43ab-bbcf-5f045a6501d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:08.426611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9adfae-1af4-43ab-bbcf-5f045a6501d6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:08.427131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9adfae-1af4-43ab-bbcf-5f045a6501d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:08.427621 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9adfae-1af4-43ab-bbcf-5f045a6501d6 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:08.428618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9adfae-1af4-43ab-bbcf-5f045a6501d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:08.431028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9adfae-1af4-43ab-bbcf-5f045a6501d6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:08.431190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:08.440159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 445/905] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 6 14:09:08 2022] GET /identity/v3/services => generated 1268 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:09:08.569333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d92a8-1002-407b-83e5-f6ec2850a26c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:08.614008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a268e-24c8-4d9b-9e58-1c2cf3756495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:08.614707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a268e-24c8-4d9b-9e58-1c2cf3756495 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:08.614991 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a268e-24c8-4d9b-9e58-1c2cf3756495 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:08.615269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a268e-24c8-4d9b-9e58-1c2cf3756495 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:08.616004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a268e-24c8-4d9b-9e58-1c2cf3756495 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:08.616310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a268e-24c8-4d9b-9e58-1c2cf3756495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:08.618815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a268e-24c8-4d9b-9e58-1c2cf3756495 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:08.625192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 461/906] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:08.631633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9adfae-1af4-43ab-bbcf-5f045a6501d6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:08.651152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:08.756333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9adfae-1af4-43ab-bbcf-5f045a6501d6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:08.797758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fc997-591d-47bc-a2a4-d43f0956a198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:08.798765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fc997-591d-47bc-a2a4-d43f0956a198 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:08.799267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fc997-591d-47bc-a2a4-d43f0956a198 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:08.799737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fc997-591d-47bc-a2a4-d43f0956a198 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:08.800676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fc997-591d-47bc-a2a4-d43f0956a198 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:08.801197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fc997-591d-47bc-a2a4-d43f0956a198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:08.802640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fc997-591d-47bc-a2a4-d43f0956a198 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:08.809278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 446/907] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:08.815640 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a268e-24c8-4d9b-9e58-1c2cf3756495 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:08.900388 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:08.937994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a268e-24c8-4d9b-9e58-1c2cf3756495 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:08.976518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1108a-7403-44f2-a4a4-30927d3905a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:08.977321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1108a-7403-44f2-a4a4-30927d3905a5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:08.977608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1108a-7403-44f2-a4a4-30927d3905a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:08.978014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1108a-7403-44f2-a4a4-30927d3905a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:08.978734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1108a-7403-44f2-a4a4-30927d3905a5 None admin] RBAC: Adding query filter params (interface=public, service_id=2d06e455ef7b40a68ec8952ee9d0ea34, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:08.979051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1108a-7403-44f2-a4a4-30927d3905a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:08.980473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1108a-7403-44f2-a4a4-30927d3905a5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:08.989806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 462/908] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 6 14:09:08 2022] GET /identity/v3/endpoints?service_id=2d06e455ef7b40a68ec8952ee9d0ea34&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:08.996806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fc997-591d-47bc-a2a4-d43f0956a198 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:09.008923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc8469-da0c-4f0f-b697-b6f4c3f543f8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:09.009414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc8469-da0c-4f0f-b697-b6f4c3f543f8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:09.009949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc8469-da0c-4f0f-b697-b6f4c3f543f8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:09.014589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 463/909] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:09.021631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41b2fe-2ebd-4b8a-acdd-3c9f6d5745f9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:09.022151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41b2fe-2ebd-4b8a-acdd-3c9f6d5745f9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:09.022624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41b2fe-2ebd-4b8a-acdd-3c9f6d5745f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:09.050877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9b41b2fe-2ebd-4b8a-acdd-3c9f6d5745f9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:09.052930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9b41b2fe-2ebd-4b8a-acdd-3c9f6d5745f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:09.107166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:09.150154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fc997-591d-47bc-a2a4-d43f0956a198 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:09.181815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 464/910] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:09.182906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45a0e9-3142-42ac-8716-09fb3f7770d3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:09.183412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45a0e9-3142-42ac-8716-09fb3f7770d3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:09.183917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45a0e9-3142-42ac-8716-09fb3f7770d3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:09.192944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 465/911] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:09.195501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9c567-80d0-4cf2-9bab-4dd4e0993d38 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:09.196020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9c567-80d0-4cf2-9bab-4dd4e0993d38 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:09.196523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9c567-80d0-4cf2-9bab-4dd4e0993d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:09.220106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9164d-3094-45b2-a47a-82b113e519b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:09.221647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9164d-3094-45b2-a47a-82b113e519b7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:09.222393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9164d-3094-45b2-a47a-82b113e519b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:09.223119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9164d-3094-45b2-a47a-82b113e519b7 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:09.224235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3cf9c567-80d0-4cf2-9bab-4dd4e0993d38 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:09.224948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9164d-3094-45b2-a47a-82b113e519b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:09.226371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3cf9c567-80d0-4cf2-9bab-4dd4e0993d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:09.228122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9164d-3094-45b2-a47a-82b113e519b7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:09.245765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 447/912] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 6 14:09:08 2022] GET /identity/v3/services => generated 1268 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:09:09.245765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9164d-3094-45b2-a47a-82b113e519b7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:09.360385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:09.382277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 466/913] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:09.423714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9c567-80d0-4cf2-9bab-4dd4e0993d38 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:09.493366 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9164d-3094-45b2-a47a-82b113e519b7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:09.560570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99edf84-b974-4603-becf-29bbcbd829b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:09.561658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99edf84-b974-4603-becf-29bbcbd829b0 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:09.562803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99edf84-b974-4603-becf-29bbcbd829b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:09.562803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99edf84-b974-4603-becf-29bbcbd829b0 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:09.563507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99edf84-b974-4603-becf-29bbcbd829b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:09.565123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99edf84-b974-4603-becf-29bbcbd829b0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:09.590271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:09.599497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 448/914] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:09:09 2022] POST /identity/v3/services => generated 241 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:09:09.648128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9c567-80d0-4cf2-9bab-4dd4e0993d38 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:09.732118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8513da-edd1-40fc-97a2-a5697338445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:09.733600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8513da-edd1-40fc-97a2-a5697338445a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:09.734121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8513da-edd1-40fc-97a2-a5697338445a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:09.734590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8513da-edd1-40fc-97a2-a5697338445a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:09.756529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8513da-edd1-40fc-97a2-a5697338445a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-9a8513da-edd1-40fc-97a2-a5697338445a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:09:09.773971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:09.776885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:09.778552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 467/915] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 6 14:09:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:09.787448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99edf84-b974-4603-becf-29bbcbd829b0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:09.987303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99edf84-b974-4603-becf-29bbcbd829b0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:10.051319 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c98ff-e372-44ea-a1ec-bfbf45be56de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:10.052017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c98ff-e372-44ea-a1ec-bfbf45be56de None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:10.052270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c98ff-e372-44ea-a1ec-bfbf45be56de None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:10.052550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c98ff-e372-44ea-a1ec-bfbf45be56de None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:10.053347 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c98ff-e372-44ea-a1ec-bfbf45be56de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:10.053633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c98ff-e372-44ea-a1ec-bfbf45be56de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:10.055923 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c98ff-e372-44ea-a1ec-bfbf45be56de None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:10.066631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 449/916] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 6 14:09:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:10.079668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8513da-edd1-40fc-97a2-a5697338445a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:10.181819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:10.280304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8513da-edd1-40fc-97a2-a5697338445a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:10.345590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc177545-23a0-4e7f-9827-6b8abe95bf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:10.347024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc177545-23a0-4e7f-9827-6b8abe95bf97 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:10.347543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc177545-23a0-4e7f-9827-6b8abe95bf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:10.348052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc177545-23a0-4e7f-9827-6b8abe95bf97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:10.371198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc177545-23a0-4e7f-9827-6b8abe95bf97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:10.373990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc177545-23a0-4e7f-9827-6b8abe95bf97 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-cc177545-23a0-4e7f-9827-6b8abe95bf97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:10.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:10.412883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:10.416761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:10.416761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:10.416761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:10.416761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:10.416761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 468/917] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:10.422886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c98ff-e372-44ea-a1ec-bfbf45be56de None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:10.502243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:10.553684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c98ff-e372-44ea-a1ec-bfbf45be56de None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:10.588248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48e497-4a9c-4623-9699-fd96ca238a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:10.588933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48e497-4a9c-4623-9699-fd96ca238a55 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:10.589219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48e497-4a9c-4623-9699-fd96ca238a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:10.589486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48e497-4a9c-4623-9699-fd96ca238a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:10.590316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48e497-4a9c-4623-9699-fd96ca238a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:10.590947 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48e497-4a9c-4623-9699-fd96ca238a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:10.592372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48e497-4a9c-4623-9699-fd96ca238a55 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:10.609162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 450/918] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:10.619106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc177545-23a0-4e7f-9827-6b8abe95bf97 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:10.736806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc177545-23a0-4e7f-9827-6b8abe95bf97 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:10.738475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:10.778838 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4070d9-d7ed-4ad9-9272-83eb95462e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:10.779716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4070d9-d7ed-4ad9-9272-83eb95462e95 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:10.779984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4070d9-d7ed-4ad9-9272-83eb95462e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:10.780257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4070d9-d7ed-4ad9-9272-83eb95462e95 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:10.788974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4070d9-d7ed-4ad9-9272-83eb95462e95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:10.791014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4070d9-d7ed-4ad9-9272-83eb95462e95 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-df4070d9-d7ed-4ad9-9272-83eb95462e95 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:09:10.809670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:10.812876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:10.814964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 469/919] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:09:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:10.822112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48e497-4a9c-4623-9699-fd96ca238a55 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:11.036083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48e497-4a9c-4623-9699-fd96ca238a55 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:11.080639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:11.100665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc20fc6-d45e-4d19-bafc-0e0287f0b322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:11.102282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc20fc6-d45e-4d19-bafc-0e0287f0b322 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:11.103438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc20fc6-d45e-4d19-bafc-0e0287f0b322 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:11.103438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc20fc6-d45e-4d19-bafc-0e0287f0b322 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:11.104752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc20fc6-d45e-4d19-bafc-0e0287f0b322 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:11.105279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc20fc6-d45e-4d19-bafc-0e0287f0b322 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:11.107955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc20fc6-d45e-4d19-bafc-0e0287f0b322 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:11.143282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 451/920] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 6 14:09:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:11.149871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4070d9-d7ed-4ad9-9272-83eb95462e95 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:11.322869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4070d9-d7ed-4ad9-9272-83eb95462e95 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:11.373187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933ac19-b85d-4759-aa54-6ac2bdfbc7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:11.374148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933ac19-b85d-4759-aa54-6ac2bdfbc7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:11.374594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933ac19-b85d-4759-aa54-6ac2bdfbc7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:11.374880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933ac19-b85d-4759-aa54-6ac2bdfbc7f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:11.390536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933ac19-b85d-4759-aa54-6ac2bdfbc7f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:11.392207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933ac19-b85d-4759-aa54-6ac2bdfbc7f4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0933ac19-b85d-4759-aa54-6ac2bdfbc7f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:11.416871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:11.418697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:11.421116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:11.421116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:11.421116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:11.421116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:11.421116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 470/921] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:11.426777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc20fc6-d45e-4d19-bafc-0e0287f0b322 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:11.620431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc20fc6-d45e-4d19-bafc-0e0287f0b322 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:11.681172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fd98f-01ba-4f5e-9c3c-b182e42fa22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:11.682279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fd98f-01ba-4f5e-9c3c-b182e42fa22a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:11.682784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fd98f-01ba-4f5e-9c3c-b182e42fa22a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:11.683278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fd98f-01ba-4f5e-9c3c-b182e42fa22a None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:11.684602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fd98f-01ba-4f5e-9c3c-b182e42fa22a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:11.684921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fd98f-01ba-4f5e-9c3c-b182e42fa22a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:11.687209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fd98f-01ba-4f5e-9c3c-b182e42fa22a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:11.708622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 452/922] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:11.715248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933ac19-b85d-4759-aa54-6ac2bdfbc7f4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:11.752186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:11.910107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933ac19-b85d-4759-aa54-6ac2bdfbc7f4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:11.975320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f435b-baac-4418-b437-54a42d337cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:11.976932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f435b-baac-4418-b437-54a42d337cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:11.977167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f435b-baac-4418-b437-54a42d337cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:11.977559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f435b-baac-4418-b437-54a42d337cd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:11.987461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f435b-baac-4418-b437-54a42d337cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:11.989448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f435b-baac-4418-b437-54a42d337cd9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2c3f435b-baac-4418-b437-54a42d337cd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:09:12.009376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:12.010417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:12.011337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:09:12.011337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:09:12.011337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:09:12.011337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:09:12.011337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:09:12.011337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:09:12.011337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:12.011337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:12.012288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 471/923] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:12.018725 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fd98f-01ba-4f5e-9c3c-b182e42fa22a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:12.069500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea90e8e-4119-49ad-b8c4-e745b28d33bd None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.072483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea90e8e-4119-49ad-b8c4-e745b28d33bd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.074542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea90e8e-4119-49ad-b8c4-e745b28d33bd None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.078988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 472/924] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:12 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:12.085794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cc716-f53f-4461-bfd9-d86f945326cd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.087291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cc716-f53f-4461-bfd9-d86f945326cd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.087841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cc716-f53f-4461-bfd9-d86f945326cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.092866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:12.108269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1d6cc716-f53f-4461-bfd9-d86f945326cd None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:12.110119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1d6cc716-f53f-4461-bfd9-d86f945326cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:12.197548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 473/925] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:12.199948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fd98f-01ba-4f5e-9c3c-b182e42fa22a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:12.227692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cc716-f53f-4461-bfd9-d86f945326cd None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:12.231802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:12.265474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ff7b-8d3c-43a7-946f-6ba8ceb4a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:12.266763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ff7b-8d3c-43a7-946f-6ba8ceb4a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.267322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ff7b-8d3c-43a7-946f-6ba8ceb4a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.267835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ff7b-8d3c-43a7-946f-6ba8ceb4a1d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.268887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0ff7b-8d3c-43a7-946f-6ba8ceb4a1d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:12.269421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0ff7b-8d3c-43a7-946f-6ba8ceb4a1d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:12.271389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0ff7b-8d3c-43a7-946f-6ba8ceb4a1d1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:12.286740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 453/926] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:09:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:12.293303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ff7b-8d3c-43a7-946f-6ba8ceb4a1d1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:12.368924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cc716-f53f-4461-bfd9-d86f945326cd None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:12.421395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce3e1a-2324-445d-b020-f7f890151ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:12.422732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce3e1a-2324-445d-b020-f7f890151ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.423192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce3e1a-2324-445d-b020-f7f890151ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.423459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce3e1a-2324-445d-b020-f7f890151ed4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.424298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce3e1a-2324-445d-b020-f7f890151ed4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:12.425783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce3e1a-2324-445d-b020-f7f890151ed4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-63ce3e1a-2324-445d-b020-f7f890151ed4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:12.439622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:12.441294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 14:09:12.443525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:12.445998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 474/927] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:09:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:12.447496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a132-f152-41ae-be8c-4493c205b07b None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.448041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a132-f152-41ae-be8c-4493c205b07b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.448041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a132-f152-41ae-be8c-4493c205b07b None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.449580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 475/928] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:12.450876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a132-f152-41ae-be8c-4493c205b07b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:12.475630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:12.482300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ff7b-8d3c-43a7-946f-6ba8ceb4a1d1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:12.539382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:12.541435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63f18f-0ec6-4fda-9dbc-40d8443e72fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:12.542356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63f18f-0ec6-4fda-9dbc-40d8443e72fe None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.542787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63f18f-0ec6-4fda-9dbc-40d8443e72fe None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.543122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63f18f-0ec6-4fda-9dbc-40d8443e72fe None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/bb0da8665c1b44f79611129ee1337bdc/roles/e57912abab304c40b99b65e14077a804` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.549423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63f18f-0ec6-4fda-9dbc-40d8443e72fe None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=bb0da8665c1b44f79611129ee1337bdc, role_id=e57912abab304c40b99b65e14077a804)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:12.549423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63f18f-0ec6-4fda-9dbc-40d8443e72fe None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:12.577737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 454/929] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:09:12 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/bb0da8665c1b44f79611129ee1337bdc/roles/e57912abab304c40b99b65e14077a804 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:09:12.577737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fd068-4587-4bd1-9ba3-371f7df80e48 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.577737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fd068-4587-4bd1-9ba3-371f7df80e48 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.577737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fd068-4587-4bd1-9ba3-371f7df80e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.588810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a132-f152-41ae-be8c-4493c205b07b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:12.603899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-812fd068-4587-4bd1-9ba3-371f7df80e48 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:12.605890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-812fd068-4587-4bd1-9ba3-371f7df80e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:12.636736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b722a-1c29-4faf-af8e-51328294a1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:12.638222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b722a-1c29-4faf-af8e-51328294a1ad None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.638529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b722a-1c29-4faf-af8e-51328294a1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.638804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b722a-1c29-4faf-af8e-51328294a1ad None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.639938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b722a-1c29-4faf-af8e-51328294a1ad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:12.640301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b722a-1c29-4faf-af8e-51328294a1ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:12.642489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b722a-1c29-4faf-af8e-51328294a1ad None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:12.650020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 476/930] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:09:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:12.656188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b722a-1c29-4faf-af8e-51328294a1ad None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:12.717021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 455/931] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:12.741360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fd068-4587-4bd1-9ba3-371f7df80e48 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:12.820939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b722a-1c29-4faf-af8e-51328294a1ad None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:12.866131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:12.870448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da115f3e-8d17-4bcf-b5ee-3c620f2b3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:12.871144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da115f3e-8d17-4bcf-b5ee-3c620f2b3a7f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.871766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da115f3e-8d17-4bcf-b5ee-3c620f2b3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.872054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da115f3e-8d17-4bcf-b5ee-3c620f2b3a7f None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.872787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da115f3e-8d17-4bcf-b5ee-3c620f2b3a7f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:12.873106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da115f3e-8d17-4bcf-b5ee-3c620f2b3a7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:12.874866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da115f3e-8d17-4bcf-b5ee-3c620f2b3a7f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:12.882194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 477/932] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 6 14:09:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:12.891944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da115f3e-8d17-4bcf-b5ee-3c620f2b3a7f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:12.910393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:12.923783 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fd068-4587-4bd1-9ba3-371f7df80e48 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:12.968727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ec81b-12f5-4d17-b4a9-36e3fc2c726a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:12.969558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec81b-12f5-4d17-b4a9-36e3fc2c726a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:12.969879 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec81b-12f5-4d17-b4a9-36e3fc2c726a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:12.970163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec81b-12f5-4d17-b4a9-36e3fc2c726a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:12.970845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ec81b-12f5-4d17-b4a9-36e3fc2c726a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:12.972248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ec81b-12f5-4d17-b4a9-36e3fc2c726a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d80ec81b-12f5-4d17-b4a9-36e3fc2c726a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:12.994133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:12.995127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:12.996155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 456/933] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Jun 6 14:09:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:13.003520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ec81b-12f5-4d17-b4a9-36e3fc2c726a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:13.066763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da115f3e-8d17-4bcf-b5ee-3c620f2b3a7f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:13.124819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ec81b-12f5-4d17-b4a9-36e3fc2c726a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:13.141053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf8253-c5d3-43b2-895d-dafa78078843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:13.142218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf8253-c5d3-43b2-895d-dafa78078843 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:13.142714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf8253-c5d3-43b2-895d-dafa78078843 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:13.143014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf8253-c5d3-43b2-895d-dafa78078843 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:13.144451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf8253-c5d3-43b2-895d-dafa78078843 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:13.147167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf8253-c5d3-43b2-895d-dafa78078843 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:13.177876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 478/934] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 6 14:09:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:09:13.177876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69865c90-d061-4e64-883d-97d4b414ab5d None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:13.177876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69865c90-d061-4e64-883d-97d4b414ab5d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:13.177876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69865c90-d061-4e64-883d-97d4b414ab5d None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:13.178735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 479/935] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:13.186387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa130dd-2c08-4a90-930c-abfeec3b3112 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:13.186783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa130dd-2c08-4a90-930c-abfeec3b3112 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:13.187021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa130dd-2c08-4a90-930c-abfeec3b3112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:13.196492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb255f-fe71-4bf3-8397-fa5268fd00a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:13.197527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb255f-fe71-4bf3-8397-fa5268fd00a8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:13.198031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb255f-fe71-4bf3-8397-fa5268fd00a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:13.198496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb255f-fe71-4bf3-8397-fa5268fd00a8 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:13.199692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb255f-fe71-4bf3-8397-fa5268fd00a8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:13.200195 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb255f-fe71-4bf3-8397-fa5268fd00a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:13.202723 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb255f-fe71-4bf3-8397-fa5268fd00a8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:13.207070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6fa130dd-2c08-4a90-930c-abfeec3b3112 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:13.208343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6fa130dd-2c08-4a90-930c-abfeec3b3112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:13.211592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 457/936] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:13.220243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb255f-fe71-4bf3-8397-fa5268fd00a8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:13.251147 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:13.280794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:13.308691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 480/937] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:13.345657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa130dd-2c08-4a90-930c-abfeec3b3112 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:13.413645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb255f-fe71-4bf3-8397-fa5268fd00a8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:13.478056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9c4ad-0f34-4186-a455-96a13675d2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:13.479212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9c4ad-0f34-4186-a455-96a13675d2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:13.484436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9c4ad-0f34-4186-a455-96a13675d2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:13.484436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9c4ad-0f34-4186-a455-96a13675d2a5 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:13.484436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9c4ad-0f34-4186-a455-96a13675d2a5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:13.484436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9c4ad-0f34-4186-a455-96a13675d2a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:13.485751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9c4ad-0f34-4186-a455-96a13675d2a5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:13.499703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 458/938] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:13.501133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9c4ad-0f34-4186-a455-96a13675d2a5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:13.551627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa130dd-2c08-4a90-930c-abfeec3b3112 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:13.619264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084443be-d57e-40f3-81aa-87b3adff49e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:13.620946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084443be-d57e-40f3-81aa-87b3adff49e5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:13.621761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084443be-d57e-40f3-81aa-87b3adff49e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:13.622560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084443be-d57e-40f3-81aa-87b3adff49e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:13.623964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084443be-d57e-40f3-81aa-87b3adff49e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:13.627112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084443be-d57e-40f3-81aa-87b3adff49e5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:13.637805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-084443be-d57e-40f3-81aa-87b3adff49e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:13.654119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:13.656021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:13.657768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:13.663494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 481/939] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:13.669803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9c4ad-0f34-4186-a455-96a13675d2a5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:13.674293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084443be-d57e-40f3-81aa-87b3adff49e5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:13.707203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ec8a0-f2d0-4293-836c-57f6bd97aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:13.708261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ec8a0-f2d0-4293-836c-57f6bd97aa2d None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:13.708261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ec8a0-f2d0-4293-836c-57f6bd97aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:13.708509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ec8a0-f2d0-4293-836c-57f6bd97aa2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:13.709363 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ec8a0-f2d0-4293-836c-57f6bd97aa2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:13.711824 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ec8a0-f2d0-4293-836c-57f6bd97aa2d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:13.745852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 459/940] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 6 14:09:13 2022] POST /identity/v3/endpoints => generated 347 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:09:13.789414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:13.910675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084443be-d57e-40f3-81aa-87b3adff49e5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:13.971537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448f8fd-72c9-4bfd-87dd-f1c55afae23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:13.972626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448f8fd-72c9-4bfd-87dd-f1c55afae23d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:13.973131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448f8fd-72c9-4bfd-87dd-f1c55afae23d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:13.973616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448f8fd-72c9-4bfd-87dd-f1c55afae23d None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:13.974633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448f8fd-72c9-4bfd-87dd-f1c55afae23d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:13.975217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448f8fd-72c9-4bfd-87dd-f1c55afae23d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:13.977452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448f8fd-72c9-4bfd-87dd-f1c55afae23d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:13.984471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 482/941] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:13.994198 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ec8a0-f2d0-4293-836c-57f6bd97aa2d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:14.190053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ec8a0-f2d0-4293-836c-57f6bd97aa2d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:14.251079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a485d3c-b332-4567-8bec-24f5541d493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:14.252099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a485d3c-b332-4567-8bec-24f5541d493e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:14.252531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a485d3c-b332-4567-8bec-24f5541d493e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:14.252962 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a485d3c-b332-4567-8bec-24f5541d493e None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:14.254189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a485d3c-b332-4567-8bec-24f5541d493e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:14.254661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a485d3c-b332-4567-8bec-24f5541d493e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:14.256977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a485d3c-b332-4567-8bec-24f5541d493e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:14.259821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:14.274772 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 460/942] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:14.278058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448f8fd-72c9-4bfd-87dd-f1c55afae23d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:14.468153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448f8fd-72c9-4bfd-87dd-f1c55afae23d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:14.481814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:14.548828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717470c7-a87d-4bd6-82b8-6220ae994683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:14.548828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717470c7-a87d-4bd6-82b8-6220ae994683 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:14.548828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717470c7-a87d-4bd6-82b8-6220ae994683 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:14.548828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717470c7-a87d-4bd6-82b8-6220ae994683 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:14.551110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717470c7-a87d-4bd6-82b8-6220ae994683 None admin] RBAC: Adding query filter params (interface=public, service_id=b8a04b163479477286d21ad0194281eb, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:14.551110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717470c7-a87d-4bd6-82b8-6220ae994683 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:14.554927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717470c7-a87d-4bd6-82b8-6220ae994683 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:14.564335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 483/943] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 6 14:09:14 2022] GET /identity/v3/endpoints?service_id=b8a04b163479477286d21ad0194281eb&interface=public®ion_id=RegionOne => generated 194 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:14.574033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a485d3c-b332-4567-8bec-24f5541d493e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:14.700665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a485d3c-b332-4567-8bec-24f5541d493e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:14.738383 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70dde9-9b54-44ce-a11f-134c0c3a2c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:14.739415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70dde9-9b54-44ce-a11f-134c0c3a2c9b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:14.739912 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70dde9-9b54-44ce-a11f-134c0c3a2c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:14.740407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70dde9-9b54-44ce-a11f-134c0c3a2c9b None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:14.741302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70dde9-9b54-44ce-a11f-134c0c3a2c9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:14.743115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70dde9-9b54-44ce-a11f-134c0c3a2c9b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:14.750646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 461/944] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 6 14:09:14 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:09:15.425978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832be4a-62a5-4bca-868d-b31a23b2d8c1 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:15.426278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832be4a-62a5-4bca-868d-b31a23b2d8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:15.426533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832be4a-62a5-4bca-868d-b31a23b2d8c1 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:15.428498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 484/945] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:15.434727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b5481-719b-4125-abe1-2876c468c9b2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:15.435337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b5481-719b-4125-abe1-2876c468c9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:15.435839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b5481-719b-4125-abe1-2876c468c9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:15.455852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3f7b5481-719b-4125-abe1-2876c468c9b2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:15.457637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3f7b5481-719b-4125-abe1-2876c468c9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:15.559238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:15.579156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 462/946] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:15.605311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832be4a-62a5-4bca-868d-b31a23b2d8c1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:15.745479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832be4a-62a5-4bca-868d-b31a23b2d8c1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:15.782763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093b069-2fe6-4232-aca6-4eb85942ecb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:15.783595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093b069-2fe6-4232-aca6-4eb85942ecb7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:15.783877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093b069-2fe6-4232-aca6-4eb85942ecb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:15.784154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093b069-2fe6-4232-aca6-4eb85942ecb7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:15.797189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093b069-2fe6-4232-aca6-4eb85942ecb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-e093b069-2fe6-4232-aca6-4eb85942ecb7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:09:15.806337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:15.807384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:15.808253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 485/947] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 6 14:09:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:15.815560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b5481-719b-4125-abe1-2876c468c9b2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:15.852569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c5a29-4c4b-43e0-8776-bf44eb445a9d None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:15.852879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c5a29-4c4b-43e0-8776-bf44eb445a9d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:15.854970 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c5a29-4c4b-43e0-8776-bf44eb445a9d None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:15.854970 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 486/948] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:15.862696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a2ff7-9ba6-4aab-9a15-ad51fa4fb2dd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:15.863037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a2ff7-9ba6-4aab-9a15-ad51fa4fb2dd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:15.863253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a2ff7-9ba6-4aab-9a15-ad51fa4fb2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:15.886645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-568a2ff7-9ba6-4aab-9a15-ad51fa4fb2dd None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:15.888269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-568a2ff7-9ba6-4aab-9a15-ad51fa4fb2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:15.928120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:15.935402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b5481-719b-4125-abe1-2876c468c9b2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:15.982284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247e17e-2409-4ef0-86da-be7d395af6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:15.983370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247e17e-2409-4ef0-86da-be7d395af6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:15.983831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247e17e-2409-4ef0-86da-be7d395af6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:15.984249 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247e17e-2409-4ef0-86da-be7d395af6c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:15.989797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 487/949] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:15.989991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247e17e-2409-4ef0-86da-be7d395af6c0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:15.989991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247e17e-2409-4ef0-86da-be7d395af6c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:15.989991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247e17e-2409-4ef0-86da-be7d395af6c0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:16.002216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 463/950] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 6 14:09:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:16.009766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a2ff7-9ba6-4aab-9a15-ad51fa4fb2dd None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:16.012813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247e17e-2409-4ef0-86da-be7d395af6c0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:16.118174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:16.164884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247e17e-2409-4ef0-86da-be7d395af6c0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:16.183461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a2ff7-9ba6-4aab-9a15-ad51fa4fb2dd None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:16.227683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:16.228881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:16.229377 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:16.229904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:16.230346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c070571-e595-407d-b5a1-4a96c8840e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:16.231224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c070571-e595-407d-b5a1-4a96c8840e24 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:16.231426 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:16.231624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c070571-e595-407d-b5a1-4a96c8840e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:16.231971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c070571-e595-407d-b5a1-4a96c8840e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:16.235029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:16.238459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WARNING keystone.server.flask.common [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 06 14:09:16.246922 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c070571-e595-407d-b5a1-4a96c8840e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:16.248616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c070571-e595-407d-b5a1-4a96c8840e24 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:16.275397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.sql.core [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 14:09:16.275397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 06 14:09:16.275397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [parameters: {'id': '320004f29a1b45da9349b65dc3a54f46', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 06 14:09:16.275397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80214) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0c070571-e595-407d-b5a1-4a96c8840e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:16.282526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:16.283560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:16.284475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:16.284475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:16.284475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:16.284475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:16.290321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 488/951] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:16.295617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c070571-e595-407d-b5a1-4a96c8840e24 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 320004f29a1b45da9349b65dc3a54f46.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 320004f29a1b45da9349b65dc3a54f46. Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application conn.query(q) Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 14:09:16.299934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application result.read() Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 06 14:09:16.300991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application next(self.gen) Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application yield resource Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application next(self.gen) Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise self.value Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application session.commit() Jun 06 14:09:16.302138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.session.flush() Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application compat.raise_( Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 14:09:16.303056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 14:09:16.304205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application conn.query(q) Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application result.read() Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 14:09:16.305950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [parameters: {'id': '320004f29a1b45da9349b65dc3a54f46', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:16.306875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:16.307903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:16.308978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:16.308978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 06 14:09:16.308978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 06 14:09:16.308978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:16.308978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:16.308978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 06 14:09:16.308978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 06 14:09:16.308978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 320004f29a1b45da9349b65dc3a54f46. Jun 06 14:09:16.308978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:16.316660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 464/952] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Jun 6 14:09:16 2022] POST /identity/v3/projects => generated 262 bytes in 301 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:16.318782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:16.320021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:16.428681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c070571-e595-407d-b5a1-4a96c8840e24 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:16.446390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f227d-9a3f-4b11-bb1c-eea6db307c5b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:16.460148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:16.479205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4088f45-cdfd-4c73-94b1-74772a2ee4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:16.480326 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4088f45-cdfd-4c73-94b1-74772a2ee4ca None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:16.480614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4088f45-cdfd-4c73-94b1-74772a2ee4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:16.480894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4088f45-cdfd-4c73-94b1-74772a2ee4ca None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:16.481647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4088f45-cdfd-4c73-94b1-74772a2ee4ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:16.481996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4088f45-cdfd-4c73-94b1-74772a2ee4ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:16.483557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4088f45-cdfd-4c73-94b1-74772a2ee4ca None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:16.493838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15644e97-bb17-4a98-b8be-355216f6ff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:16.494943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15644e97-bb17-4a98-b8be-355216f6ff95 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:16.495370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15644e97-bb17-4a98-b8be-355216f6ff95 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:16.495784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15644e97-bb17-4a98-b8be-355216f6ff95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:16.503981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15644e97-bb17-4a98-b8be-355216f6ff95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:16.506064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15644e97-bb17-4a98-b8be-355216f6ff95 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:16.509612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 489/953] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:16.510563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4088f45-cdfd-4c73-94b1-74772a2ee4ca None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-15644e97-bb17-4a98-b8be-355216f6ff95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:09:16.522060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:16.523041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:16.524617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:09:16.524617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:09:16.524617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:09:16.524617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:09:16.524617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:09:16.524617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:09:16.524617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:16.524617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:16.532016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 465/954] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:16.533636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15644e97-bb17-4a98-b8be-355216f6ff95 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:16.647964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:16.690734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4088f45-cdfd-4c73-94b1-74772a2ee4ca None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:16.700425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15644e97-bb17-4a98-b8be-355216f6ff95 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:16.735432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26da1d-e5d7-47a9-afd8-72e7bcf8985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:16.736375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26da1d-e5d7-47a9-afd8-72e7bcf8985c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:16.736663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26da1d-e5d7-47a9-afd8-72e7bcf8985c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:16.736946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26da1d-e5d7-47a9-afd8-72e7bcf8985c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:16.744274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35d0b5-f2cb-49ba-9001-b25502532b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:16.744943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35d0b5-f2cb-49ba-9001-b25502532b9e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:16.745219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35d0b5-f2cb-49ba-9001-b25502532b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:16.745495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35d0b5-f2cb-49ba-9001-b25502532b9e None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:16.746281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35d0b5-f2cb-49ba-9001-b25502532b9e None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:16.746586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35d0b5-f2cb-49ba-9001-b25502532b9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:16.747093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26da1d-e5d7-47a9-afd8-72e7bcf8985c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:16.748012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35d0b5-f2cb-49ba-9001-b25502532b9e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:16.748730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26da1d-e5d7-47a9-afd8-72e7bcf8985c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1f26da1d-e5d7-47a9-afd8-72e7bcf8985c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:09:16.765850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:16.767747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 14:09:16.770171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:16.772274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 490/955] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:09:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:16.774293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dde5cb-b213-42f7-96e0-a8765e889f5e None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:16.774580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dde5cb-b213-42f7-96e0-a8765e889f5e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:16.774840 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dde5cb-b213-42f7-96e0-a8765e889f5e None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:16.775376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 466/956] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:09:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:16.776499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 491/957] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:16.777681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35d0b5-f2cb-49ba-9001-b25502532b9e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:16.781851 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0dc6a-924a-4deb-8ad5-588ab015923e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:16.782215 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0dc6a-924a-4deb-8ad5-588ab015923e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:16.782409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0dc6a-924a-4deb-8ad5-588ab015923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:16.803587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-89e0dc6a-924a-4deb-8ad5-588ab015923e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:16.805412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-89e0dc6a-924a-4deb-8ad5-588ab015923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:16.821152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:16.905843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 492/958] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:16.931016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0dc6a-924a-4deb-8ad5-588ab015923e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:16.941657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35d0b5-f2cb-49ba-9001-b25502532b9e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:16.983626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58127542-bede-497f-a2a6-8feacef6b643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:16.984522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58127542-bede-497f-a2a6-8feacef6b643 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:16.984638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58127542-bede-497f-a2a6-8feacef6b643 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:16.985077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58127542-bede-497f-a2a6-8feacef6b643 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:16.985977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58127542-bede-497f-a2a6-8feacef6b643 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:16.986327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58127542-bede-497f-a2a6-8feacef6b643 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:16.987740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58127542-bede-497f-a2a6-8feacef6b643 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:17.008463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:17.009812 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 467/959] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 6 14:09:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:17.016275 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58127542-bede-497f-a2a6-8feacef6b643 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:17.058154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0dc6a-924a-4deb-8ad5-588ab015923e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:17.098253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1770d8-8c70-4ae6-a674-2218a92866b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:17.099456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1770d8-8c70-4ae6-a674-2218a92866b4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:17.099968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1770d8-8c70-4ae6-a674-2218a92866b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:17.100448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1770d8-8c70-4ae6-a674-2218a92866b4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:17.101325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1770d8-8c70-4ae6-a674-2218a92866b4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:17.103559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1770d8-8c70-4ae6-a674-2218a92866b4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-3e1770d8-8c70-4ae6-a674-2218a92866b4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:17.117261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:17.118446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 14:09:17.119356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:17.120150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:17.122661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 493/960] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:17.133956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1770d8-8c70-4ae6-a674-2218a92866b4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:17.135521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58127542-bede-497f-a2a6-8feacef6b643 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:17.184135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1be09a-3f79-4125-b9df-178a009f0a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:17.184797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1be09a-3f79-4125-b9df-178a009f0a41 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:17.185079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1be09a-3f79-4125-b9df-178a009f0a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:17.185355 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1be09a-3f79-4125-b9df-178a009f0a41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:17.284432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1770d8-8c70-4ae6-a674-2218a92866b4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:17.313593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1be09a-3f79-4125-b9df-178a009f0a41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:17.316278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1be09a-3f79-4125-b9df-178a009f0a41 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:17.336548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6cb4b-6f05-4eb6-801c-8b7234c87607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:17.338565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6cb4b-6f05-4eb6-801c-8b7234c87607 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:17.341680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6cb4b-6f05-4eb6-801c-8b7234c87607 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:17.342169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6cb4b-6f05-4eb6-801c-8b7234c87607 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:17.343531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:17.345158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6cb4b-6f05-4eb6-801c-8b7234c87607 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:17.345750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6cb4b-6f05-4eb6-801c-8b7234c87607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:17.348602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6cb4b-6f05-4eb6-801c-8b7234c87607 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:17.365900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 494/961] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:09:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:17.369745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6cb4b-6f05-4eb6-801c-8b7234c87607 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:17.456832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:17.477405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 468/962] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:09:17 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:09:17.483189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1be09a-3f79-4125-b9df-178a009f0a41 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:17.591734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6cb4b-6f05-4eb6-801c-8b7234c87607 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:17.614516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1be09a-3f79-4125-b9df-178a009f0a41 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:17.662835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1a640-e475-40e3-9f7c-23f5b1c9659c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:17.664662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8149595-24d3-4aaf-b914-6a766967f959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:17.664799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1a640-e475-40e3-9f7c-23f5b1c9659c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:17.665208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1a640-e475-40e3-9f7c-23f5b1c9659c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:17.665751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1a640-e475-40e3-9f7c-23f5b1c9659c None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:17.665854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8149595-24d3-4aaf-b914-6a766967f959 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:17.666135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8149595-24d3-4aaf-b914-6a766967f959 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:17.666405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8149595-24d3-4aaf-b914-6a766967f959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:17.667165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1a640-e475-40e3-9f7c-23f5b1c9659c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:17.667660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1a640-e475-40e3-9f7c-23f5b1c9659c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:17.670320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1a640-e475-40e3-9f7c-23f5b1c9659c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:17.678200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8149595-24d3-4aaf-b914-6a766967f959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:17.681870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 495/963] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:09:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:17.682083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8149595-24d3-4aaf-b914-6a766967f959 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:17.682169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3538f35-f9ca-48a5-8d68-d666e1aea640 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:17.682169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3538f35-f9ca-48a5-8d68-d666e1aea640 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:17.682529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3538f35-f9ca-48a5-8d68-d666e1aea640 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:17.685760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 496/964] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:17.691115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf89400-3b08-4370-a9b8-bd65d34e1c2f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:17.691463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf89400-3b08-4370-a9b8-bd65d34e1c2f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:17.691678 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf89400-3b08-4370-a9b8-bd65d34e1c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:17.713765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-fdf89400-3b08-4370-a9b8-bd65d34e1c2f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:17.715109 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fdf89400-3b08-4370-a9b8-bd65d34e1c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e8149595-24d3-4aaf-b914-6a766967f959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:17.728980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:17.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:17.733183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:17.733183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:17.733183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:17.733183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:17.737013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 469/965] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:17.743373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8149595-24d3-4aaf-b914-6a766967f959 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:17.802221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:17.818246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:17.873833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 497/966] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:17.892461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf89400-3b08-4370-a9b8-bd65d34e1c2f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:17.902795 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8149595-24d3-4aaf-b914-6a766967f959 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:17.946919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10924ba2-29e8-4cef-8a06-be4eccc54ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:17.947986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10924ba2-29e8-4cef-8a06-be4eccc54ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:17.948388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10924ba2-29e8-4cef-8a06-be4eccc54ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:17.948766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10924ba2-29e8-4cef-8a06-be4eccc54ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:17.949612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10924ba2-29e8-4cef-8a06-be4eccc54ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:17.949968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10924ba2-29e8-4cef-8a06-be4eccc54ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:17.951495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10924ba2-29e8-4cef-8a06-be4eccc54ea5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:17.966935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 470/967] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:17.972403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10924ba2-29e8-4cef-8a06-be4eccc54ea5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:18.073999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf89400-3b08-4370-a9b8-bd65d34e1c2f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:18.091307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:18.095055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10924ba2-29e8-4cef-8a06-be4eccc54ea5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:18.120643 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc747c8-19c8-4ac1-b8ff-d2aace68d482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:18.122699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc747c8-19c8-4ac1-b8ff-d2aace68d482 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:18.123610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc747c8-19c8-4ac1-b8ff-d2aace68d482 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:18.124555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc747c8-19c8-4ac1-b8ff-d2aace68d482 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:18.126556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc747c8-19c8-4ac1-b8ff-d2aace68d482 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:18.128942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc747c8-19c8-4ac1-b8ff-d2aace68d482 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:18.139768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412144f-28e9-40d5-ba68-fdeb7f6743ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:18.140584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412144f-28e9-40d5-ba68-fdeb7f6743ca None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:18.140864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412144f-28e9-40d5-ba68-fdeb7f6743ca None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:18.141139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412144f-28e9-40d5-ba68-fdeb7f6743ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:18.149967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412144f-28e9-40d5-ba68-fdeb7f6743ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:18.151481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412144f-28e9-40d5-ba68-fdeb7f6743ca None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-cdc747c8-19c8-4ac1-b8ff-d2aace68d482 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:18.156637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:18.158455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:18.159926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:18.164501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 498/968] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:18.171878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc747c8-19c8-4ac1-b8ff-d2aace68d482 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1412144f-28e9-40d5-ba68-fdeb7f6743ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:09:18.172316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:18.173248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:18.174260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:09:18.174260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:09:18.174260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:09:18.174260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:09:18.174260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:09:18.174260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:09:18.174260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:18.174260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:18.174260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 471/969] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:18.180542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412144f-28e9-40d5-ba68-fdeb7f6743ca None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:18.235651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:18.297852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc747c8-19c8-4ac1-b8ff-d2aace68d482 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:18.305792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412144f-28e9-40d5-ba68-fdeb7f6743ca None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:18.338524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b89aeb-8901-4207-af4e-8846850b463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:18.339631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b89aeb-8901-4207-af4e-8846850b463d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:18.340254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b89aeb-8901-4207-af4e-8846850b463d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:18.340757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b89aeb-8901-4207-af4e-8846850b463d None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:18.341853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b89aeb-8901-4207-af4e-8846850b463d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:18.342429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b89aeb-8901-4207-af4e-8846850b463d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:18.344605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b89aeb-8901-4207-af4e-8846850b463d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:18.346711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1b01c-09f7-4492-9e41-ff6546cc40ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:18.347418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1b01c-09f7-4492-9e41-ff6546cc40ff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:18.347699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1b01c-09f7-4492-9e41-ff6546cc40ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:18.347973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1b01c-09f7-4492-9e41-ff6546cc40ff None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:18.348915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1b01c-09f7-4492-9e41-ff6546cc40ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:18.349226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1b01c-09f7-4492-9e41-ff6546cc40ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:18.350813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1b01c-09f7-4492-9e41-ff6546cc40ff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:18.353965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 499/970] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:18.360978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b89aeb-8901-4207-af4e-8846850b463d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:18.373755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 472/971] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:09:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:18.381823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1b01c-09f7-4492-9e41-ff6546cc40ff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:18.413111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:18.538570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1b01c-09f7-4492-9e41-ff6546cc40ff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:18.541905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b89aeb-8901-4207-af4e-8846850b463d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:18.567988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:18.585502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55419d-d385-4d39-96c3-f39ecd625613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:18.587872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55419d-d385-4d39-96c3-f39ecd625613 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:18.588146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55419d-d385-4d39-96c3-f39ecd625613 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:18.588418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55419d-d385-4d39-96c3-f39ecd625613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:18.589263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55419d-d385-4d39-96c3-f39ecd625613 None admin] RBAC: Adding query filter params (role.id=e57912abab304c40b99b65e14077a804, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=bb0da8665c1b44f79611129ee1337bdc) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:18.589571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55419d-d385-4d39-96c3-f39ecd625613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:18.591388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55419d-d385-4d39-96c3-f39ecd625613 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:18.602158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 473/972] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:09:18 2022] GET /identity/v3/role_assignments?user.id=bb0da8665c1b44f79611129ee1337bdc&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=e57912abab304c40b99b65e14077a804 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:18.618927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb09bed-a6e3-4d08-8f3f-8e0aa5cd6d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:18.620773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09bed-a6e3-4d08-8f3f-8e0aa5cd6d39 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:18.621634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09bed-a6e3-4d08-8f3f-8e0aa5cd6d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:18.622523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09bed-a6e3-4d08-8f3f-8e0aa5cd6d39 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:18.624309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb09bed-a6e3-4d08-8f3f-8e0aa5cd6d39 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:18.625250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb09bed-a6e3-4d08-8f3f-8e0aa5cd6d39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:18.628674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb09bed-a6e3-4d08-8f3f-8e0aa5cd6d39 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:18.640554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 500/973] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:18.646721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55419d-d385-4d39-96c3-f39ecd625613 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:18.816667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:18.853089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55419d-d385-4d39-96c3-f39ecd625613 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:18.920542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70690c-5e9a-4dc5-aacd-872abb1080df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:18.922168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70690c-5e9a-4dc5-aacd-872abb1080df None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:18.922938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70690c-5e9a-4dc5-aacd-872abb1080df None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:18.923426 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70690c-5e9a-4dc5-aacd-872abb1080df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:18.925075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70690c-5e9a-4dc5-aacd-872abb1080df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:18.928044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70690c-5e9a-4dc5-aacd-872abb1080df None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:18.967975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 474/974] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 6 14:09:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:09:19.583391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263f66a-f985-4673-893a-bdcf81ac9ee2 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:19.583944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263f66a-f985-4673-893a-bdcf81ac9ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:19.583944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263f66a-f985-4673-893a-bdcf81ac9ee2 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:19.585623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 501/975] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:19.592072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe154c-3665-4cea-bdc3-b4fe5c760ef8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:19.592367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe154c-3665-4cea-bdc3-b4fe5c760ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:19.592651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe154c-3665-4cea-bdc3-b4fe5c760ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:19.612232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-61fe154c-3665-4cea-bdc3-b4fe5c760ef8 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:19.614002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-61fe154c-3665-4cea-bdc3-b4fe5c760ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:19.702156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 475/976] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:19.723686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263f66a-f985-4673-893a-bdcf81ac9ee2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:19.868820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263f66a-f985-4673-893a-bdcf81ac9ee2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:19.913340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:19.915054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:19.915568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:19.916060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:19.917055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:19.918882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:19.920525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING keystone.server.flask.common [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 06 14:09:19.936588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.sql.core [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 14:09:19.936588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 06 14:09:19.936588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [parameters: {'user_id': '812ed3af9b8e48f8b95bac574a775c9d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 06 14:09:19.936588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80215) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application conn.query(q) Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result.read() Jun 06 14:09:19.944956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 14:09:19.947528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application next(self.gen) Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application yield resource Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application next(self.gen) Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise self.value Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application session.commit() Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 14:09:19.959513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.session.flush() Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application compat.raise_( Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 14:09:19.962706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 14:09:19.964847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application conn.query(q) Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result.read() Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 14:09:19.966896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [parameters: {'user_id': '812ed3af9b8e48f8b95bac574a775c9d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:19.969035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 06 14:09:19.971075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 06 14:09:19.973210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 06 14:09:19.973210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 06 14:09:19.973210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 14:09:19.973210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 14:09:19.973210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 06 14:09:19.973210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:19.973210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 502/977] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 6 14:09:19 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:19.974163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe154c-3665-4cea-bdc3-b4fe5c760ef8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:20.048978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:20.093407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe154c-3665-4cea-bdc3-b4fe5c760ef8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:20.131344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872f9dd-075b-4e64-bcfd-2e76cb7d08f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:20.132070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872f9dd-075b-4e64-bcfd-2e76cb7d08f0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:20.132351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872f9dd-075b-4e64-bcfd-2e76cb7d08f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:20.133200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872f9dd-075b-4e64-bcfd-2e76cb7d08f0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:20.141435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872f9dd-075b-4e64-bcfd-2e76cb7d08f0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:20.142946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872f9dd-075b-4e64-bcfd-2e76cb7d08f0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a872f9dd-075b-4e64-bcfd-2e76cb7d08f0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:09:20.158690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:20.161054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:20.163449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 476/978] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:09:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:20.171137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:20.237874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:20.346672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbc72a-b0de-46aa-8b51-8451e34d389a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:20.408385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad373e-b031-48be-993e-23b2cd96c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:20.411546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad373e-b031-48be-993e-23b2cd96c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:20.412331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad373e-b031-48be-993e-23b2cd96c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:20.413093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad373e-b031-48be-993e-23b2cd96c8c6 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:20.415142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad373e-b031-48be-993e-23b2cd96c8c6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:20.415972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad373e-b031-48be-993e-23b2cd96c8c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:20.419471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad373e-b031-48be-993e-23b2cd96c8c6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:20.475798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 503/979] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 6 14:09:20 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:21.007130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3d774-3b51-4f49-9c33-0dcdf1cffbcc None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.008329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3d774-3b51-4f49-9c33-0dcdf1cffbcc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.008329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3d774-3b51-4f49-9c33-0dcdf1cffbcc None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.009556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 477/980] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:21.015493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd74ca-9bfb-4c54-99d4-3ce37773f273 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.016280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd74ca-9bfb-4c54-99d4-3ce37773f273 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.016280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd74ca-9bfb-4c54-99d4-3ce37773f273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.035514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-49dd74ca-9bfb-4c54-99d4-3ce37773f273 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:21.037189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-49dd74ca-9bfb-4c54-99d4-3ce37773f273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:21.127014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 504/981] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:21.150613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3d774-3b51-4f49-9c33-0dcdf1cffbcc None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:21.253839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e631f-156c-49cc-b192-ac605ee75390 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.254344 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e631f-156c-49cc-b192-ac605ee75390 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.254855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e631f-156c-49cc-b192-ac605ee75390 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.259132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 505/982] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:21.266621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590521e-a325-4922-b6ba-63a7cf860d20 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.267098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590521e-a325-4922-b6ba-63a7cf860d20 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.267551 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590521e-a325-4922-b6ba-63a7cf860d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.293934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2590521e-a325-4922-b6ba-63a7cf860d20 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:21.295862 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2590521e-a325-4922-b6ba-63a7cf860d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:21.331429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:21.369626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3d774-3b51-4f49-9c33-0dcdf1cffbcc None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:21.417737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 506/983] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:21.422554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:21.437398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590521e-a325-4922-b6ba-63a7cf860d20 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:21.439117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36215d-380d-42bf-aa11-0f6a6494c2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:21.440250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36215d-380d-42bf-aa11-0f6a6494c2c3 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.440750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36215d-380d-42bf-aa11-0f6a6494c2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.441235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36215d-380d-42bf-aa11-0f6a6494c2c3 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.442639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36215d-380d-42bf-aa11-0f6a6494c2c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:21.445935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36215d-380d-42bf-aa11-0f6a6494c2c3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:21.470076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 478/984] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:09:21 2022] POST /identity/v3/services => generated 242 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:09:21.534843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4a195-52a3-4f6c-be99-f467db84de4e None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.535126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4a195-52a3-4f6c-be99-f467db84de4e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.535402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4a195-52a3-4f6c-be99-f467db84de4e None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.537201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 479/985] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:21.543386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fd6c7-0262-481f-90f8-2c7afc615e0f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.544012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fd6c7-0262-481f-90f8-2c7afc615e0f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.544012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fd6c7-0262-481f-90f8-2c7afc615e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.563559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1d7fd6c7-0262-481f-90f8-2c7afc615e0f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:21.564860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1d7fd6c7-0262-481f-90f8-2c7afc615e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:21.628427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590521e-a325-4922-b6ba-63a7cf860d20 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:21.684225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 480/986] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:21.698770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7f173-71e9-4f12-9780-326bf22e90f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:21.700153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7f173-71e9-4f12-9780-326bf22e90f7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.700650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7f173-71e9-4f12-9780-326bf22e90f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.701137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7f173-71e9-4f12-9780-326bf22e90f7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.702435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7f173-71e9-4f12-9780-326bf22e90f7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:21.705145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7f173-71e9-4f12-9780-326bf22e90f7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:21.713573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fd6c7-0262-481f-90f8-2c7afc615e0f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-41e7f173-71e9-4f12-9780-326bf22e90f7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:21.721297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:21.723294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:21.724967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 507/987] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Jun 6 14:09:21 2022] GET /identity/v3/services/image => generated 86 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:21.731627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7f173-71e9-4f12-9780-326bf22e90f7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:21.759266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:21.853210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fd6c7-0262-481f-90f8-2c7afc615e0f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:21.869130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:21.884792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7f173-71e9-4f12-9780-326bf22e90f7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:21.898688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab649d0-3f21-4481-8bed-16a42af21c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:21.899806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab649d0-3f21-4481-8bed-16a42af21c95 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.900085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab649d0-3f21-4481-8bed-16a42af21c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.900356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab649d0-3f21-4481-8bed-16a42af21c95 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.901032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab649d0-3f21-4481-8bed-16a42af21c95 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:21.902474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab649d0-3f21-4481-8bed-16a42af21c95 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2ab649d0-3f21-4481-8bed-16a42af21c95 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:21.916044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:21.917166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:21.918079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 481/988] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:21.926904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab649d0-3f21-4481-8bed-16a42af21c95 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:21.928752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8ee02-1c69-4b32-b873-74e54243f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:21.930097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8ee02-1c69-4b32-b873-74e54243f119 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:21.930415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8ee02-1c69-4b32-b873-74e54243f119 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:21.930794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8ee02-1c69-4b32-b873-74e54243f119 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:21.931533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8ee02-1c69-4b32-b873-74e54243f119 None admin] RBAC: Adding query filter params (name=image) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:21.931841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8ee02-1c69-4b32-b873-74e54243f119 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:21.933261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8ee02-1c69-4b32-b873-74e54243f119 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:21.939781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 508/989] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:21.946675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8ee02-1c69-4b32-b873-74e54243f119 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:22.081622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab649d0-3f21-4481-8bed-16a42af21c95 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:22.087996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:22.119050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8ee02-1c69-4b32-b873-74e54243f119 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:22.122156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f041815-b872-4979-8120-e4e57937dc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:22.124128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f041815-b872-4979-8120-e4e57937dc96 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:22.124667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f041815-b872-4979-8120-e4e57937dc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:22.125155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f041815-b872-4979-8120-e4e57937dc96 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:22.126123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f041815-b872-4979-8120-e4e57937dc96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:22.126644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f041815-b872-4979-8120-e4e57937dc96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:22.128240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f041815-b872-4979-8120-e4e57937dc96 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:22.136032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 482/990] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:22.146092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f041815-b872-4979-8120-e4e57937dc96 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:22.164242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69436e-ad00-4b5d-9df6-a457a4a9f5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:22.164936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69436e-ad00-4b5d-9df6-a457a4a9f5cf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:22.165207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69436e-ad00-4b5d-9df6-a457a4a9f5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:22.165478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69436e-ad00-4b5d-9df6-a457a4a9f5cf None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:22.166758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69436e-ad00-4b5d-9df6-a457a4a9f5cf None admin] RBAC: Adding query filter params (type=image) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:22.167076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69436e-ad00-4b5d-9df6-a457a4a9f5cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:22.168889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69436e-ad00-4b5d-9df6-a457a4a9f5cf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:22.181758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 509/991] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:22.228553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:22.348067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f041815-b872-4979-8120-e4e57937dc96 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:22.413026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab086b8f-c5b6-40c1-bcbd-22616159f78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:22.414819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab086b8f-c5b6-40c1-bcbd-22616159f78e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:22.415251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab086b8f-c5b6-40c1-bcbd-22616159f78e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:22.415752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab086b8f-c5b6-40c1-bcbd-22616159f78e None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:22.417153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab086b8f-c5b6-40c1-bcbd-22616159f78e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:22.417819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab086b8f-c5b6-40c1-bcbd-22616159f78e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:22.420413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab086b8f-c5b6-40c1-bcbd-22616159f78e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:22.431713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 483/992] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:22.438134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69436e-ad00-4b5d-9df6-a457a4a9f5cf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:22.578656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69436e-ad00-4b5d-9df6-a457a4a9f5cf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:22.625476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa0d2b-4ed4-4284-abef-2cf6d39ee0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:22.626177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa0d2b-4ed4-4284-abef-2cf6d39ee0bf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:22.626460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa0d2b-4ed4-4284-abef-2cf6d39ee0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:22.626733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa0d2b-4ed4-4284-abef-2cf6d39ee0bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:22.627568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa0d2b-4ed4-4284-abef-2cf6d39ee0bf None admin] RBAC: Adding query filter params (interface=admin, service_id=b8a04b163479477286d21ad0194281eb, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:22.627880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa0d2b-4ed4-4284-abef-2cf6d39ee0bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:22.629341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa0d2b-4ed4-4284-abef-2cf6d39ee0bf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:22.636636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 510/993] 10.222.0.13 () {56 vars in 1249 bytes} [Mon Jun 6 14:09:22 2022] GET /identity/v3/endpoints?service_id=b8a04b163479477286d21ad0194281eb&interface=admin®ion_id=RegionOne => generated 193 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:22.642869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab086b8f-c5b6-40c1-bcbd-22616159f78e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:22.838257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab086b8f-c5b6-40c1-bcbd-22616159f78e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:22.842869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:22.905993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cd5c6-dfea-4798-be4e-1058707cc301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:22.907145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cd5c6-dfea-4798-be4e-1058707cc301 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:22.907655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cd5c6-dfea-4798-be4e-1058707cc301 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:22.908150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cd5c6-dfea-4798-be4e-1058707cc301 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:22.909469 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cd5c6-dfea-4798-be4e-1058707cc301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:22.912195 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cd5c6-dfea-4798-be4e-1058707cc301 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:22.934083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 484/994] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 6 14:09:22 2022] GET /identity/v3/services => generated 1727 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:09:24.409062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6f8b2-5fe2-4eed-a282-5ecfc79ff8b0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:24.409646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6f8b2-5fe2-4eed-a282-5ecfc79ff8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:24.409646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6f8b2-5fe2-4eed-a282-5ecfc79ff8b0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:24.411425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 511/995] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:24.417829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475dad8-7d56-4166-ac9e-e08233d665d5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:24.418453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475dad8-7d56-4166-ac9e-e08233d665d5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:24.418453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475dad8-7d56-4166-ac9e-e08233d665d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:24.436955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f475dad8-7d56-4166-ac9e-e08233d665d5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:24.438189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f475dad8-7d56-4166-ac9e-e08233d665d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:24.545629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 485/996] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:24.568232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6f8b2-5fe2-4eed-a282-5ecfc79ff8b0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:24.573799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4400d-de48-4cb4-88f9-414d5db7bf5b None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:24.574068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4400d-de48-4cb4-88f9-414d5db7bf5b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:24.574437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4400d-de48-4cb4-88f9-414d5db7bf5b None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:24.576033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 486/997] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:24.581911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f9c1-76d2-4bc5-8829-32714171aeb0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:24.582245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f9c1-76d2-4bc5-8829-32714171aeb0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:24.582485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f9c1-76d2-4bc5-8829-32714171aeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:24.585990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:24.604483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9971f9c1-76d2-4bc5-8829-32714171aeb0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:24.605813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9971f9c1-76d2-4bc5-8829-32714171aeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:24.727774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 487/998] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:24.754886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6f8b2-5fe2-4eed-a282-5ecfc79ff8b0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:24.768630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f9c1-76d2-4bc5-8829-32714171aeb0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:24.815013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:24.818115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7d534-1e8c-41cc-b83b-eb5eaeea5fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:24.819456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7d534-1e8c-41cc-b83b-eb5eaeea5fda None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:24.819986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7d534-1e8c-41cc-b83b-eb5eaeea5fda None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:24.821014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7d534-1e8c-41cc-b83b-eb5eaeea5fda None admin] PATH_INFO: `/v3/services/compute` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:24.822305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7d534-1e8c-41cc-b83b-eb5eaeea5fda None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:24.824786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7d534-1e8c-41cc-b83b-eb5eaeea5fda None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-5fd7d534-1e8c-41cc-b83b-eb5eaeea5fda None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:24.847355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:24.849364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:24.854043 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 512/999] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:24.859837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7d534-1e8c-41cc-b83b-eb5eaeea5fda None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:24.947392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f9c1-76d2-4bc5-8829-32714171aeb0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:25.005802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:25.010181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096de254-5cff-4541-9df6-10dca404f056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:25.011308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096de254-5cff-4541-9df6-10dca404f056 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:25.011781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096de254-5cff-4541-9df6-10dca404f056 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:25.012253 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096de254-5cff-4541-9df6-10dca404f056 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:25.013592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096de254-5cff-4541-9df6-10dca404f056 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:25.016216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096de254-5cff-4541-9df6-10dca404f056 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:25.039629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 488/1000] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:09:24 2022] POST /identity/v3/services => generated 242 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:09:25.059972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7d534-1e8c-41cc-b83b-eb5eaeea5fda None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:25.118646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06219f9d-b966-47eb-8d11-3dd465a76daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:25.119337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06219f9d-b966-47eb-8d11-3dd465a76daa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:25.119613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06219f9d-b966-47eb-8d11-3dd465a76daa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:25.119911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06219f9d-b966-47eb-8d11-3dd465a76daa None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:25.120635 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06219f9d-b966-47eb-8d11-3dd465a76daa None admin] RBAC: Adding query filter params (name=compute) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:25.120946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06219f9d-b966-47eb-8d11-3dd465a76daa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:25.122484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06219f9d-b966-47eb-8d11-3dd465a76daa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:25.129219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 513/1001] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:09:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:25.136954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096de254-5cff-4541-9df6-10dca404f056 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:25.253669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:25.271461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096de254-5cff-4541-9df6-10dca404f056 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:25.314636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e18a5-bb86-410f-87b4-6cd5737dd791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:25.315320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e18a5-bb86-410f-87b4-6cd5737dd791 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:25.315601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e18a5-bb86-410f-87b4-6cd5737dd791 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:25.315872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e18a5-bb86-410f-87b4-6cd5737dd791 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:25.316601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e18a5-bb86-410f-87b4-6cd5737dd791 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:25.316918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e18a5-bb86-410f-87b4-6cd5737dd791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:25.318393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e18a5-bb86-410f-87b4-6cd5737dd791 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:25.325544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 489/1002] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:09:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:25.331373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06219f9d-b966-47eb-8d11-3dd465a76daa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:25.515221 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06219f9d-b966-47eb-8d11-3dd465a76daa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:25.573465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ae7ac-4165-42bd-b19a-175a53343daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:25.574211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ae7ac-4165-42bd-b19a-175a53343daf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:25.574498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ae7ac-4165-42bd-b19a-175a53343daf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:25.574773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ae7ac-4165-42bd-b19a-175a53343daf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:25.575596 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ae7ac-4165-42bd-b19a-175a53343daf None admin] RBAC: Adding query filter params (interface=public, service_id=339a5f3839294e8f8b47ed6398e330e5, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:25.575908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ae7ac-4165-42bd-b19a-175a53343daf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:25.577348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ae7ac-4165-42bd-b19a-175a53343daf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:25.581276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:25.592513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 514/1003] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 6 14:09:25 2022] GET /identity/v3/endpoints?service_id=339a5f3839294e8f8b47ed6398e330e5&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:25.593955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e18a5-bb86-410f-87b4-6cd5737dd791 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:25.721565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e18a5-bb86-410f-87b4-6cd5737dd791 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:25.722042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d0d91-6c2d-4b26-a246-7e0ef347c8c3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:25.722316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d0d91-6c2d-4b26-a246-7e0ef347c8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:25.722608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d0d91-6c2d-4b26-a246-7e0ef347c8c3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:25.724252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 515/1004] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:25.732030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdd0eb-dca3-49d4-a219-817667755ddc None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:25.732433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdd0eb-dca3-49d4-a219-817667755ddc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:25.733003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdd0eb-dca3-49d4-a219-817667755ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:25.752349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-37fdd0eb-dca3-49d4-a219-817667755ddc None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:25.753579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-37fdd0eb-dca3-49d4-a219-817667755ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:25.764737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5f798-5666-4f48-b5ac-0b850ff7f6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:25.765411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5f798-5666-4f48-b5ac-0b850ff7f6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:25.765691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5f798-5666-4f48-b5ac-0b850ff7f6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:25.766012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5f798-5666-4f48-b5ac-0b850ff7f6d5 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:25.766701 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5f798-5666-4f48-b5ac-0b850ff7f6d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:25.768082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5f798-5666-4f48-b5ac-0b850ff7f6d5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:25.775583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 490/1005] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 6 14:09:25 2022] GET /identity/v3/services => generated 1957 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:09:25.842593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 516/1006] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:25 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:25.867219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5f798-5666-4f48-b5ac-0b850ff7f6d5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:25.897340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:26.041858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5f798-5666-4f48-b5ac-0b850ff7f6d5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:26.091272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48887-05c9-4913-9e1b-e3d842c8db64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:26.092300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48887-05c9-4913-9e1b-e3d842c8db64 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:26.092787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48887-05c9-4913-9e1b-e3d842c8db64 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:26.093148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48887-05c9-4913-9e1b-e3d842c8db64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:26.093990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e48887-05c9-4913-9e1b-e3d842c8db64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:26.095523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e48887-05c9-4913-9e1b-e3d842c8db64 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-98e48887-05c9-4913-9e1b-e3d842c8db64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:26.109262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:26.111677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:26.113506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 491/1007] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 6 14:09:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:26.121514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdd0eb-dca3-49d4-a219-817667755ddc None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:26.312530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdd0eb-dca3-49d4-a219-817667755ddc None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:26.369381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:26.380350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8c248-8b7a-4bda-baf9-84e40e24325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:26.381410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8c248-8b7a-4bda-baf9-84e40e24325c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:26.381907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8c248-8b7a-4bda-baf9-84e40e24325c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:26.382557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8c248-8b7a-4bda-baf9-84e40e24325c None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:26.383829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8c248-8b7a-4bda-baf9-84e40e24325c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:26.384336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8c248-8b7a-4bda-baf9-84e40e24325c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:26.387143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8c248-8b7a-4bda-baf9-84e40e24325c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:26.404746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 517/1008] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:26.406830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48887-05c9-4913-9e1b-e3d842c8db64 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:26.574515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:26.614862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48887-05c9-4913-9e1b-e3d842c8db64 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:26.687530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b0c81-f688-4a26-ab4a-6a4bf30c425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:26.688439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b0c81-f688-4a26-ab4a-6a4bf30c425f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:26.688873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b0c81-f688-4a26-ab4a-6a4bf30c425f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:26.689294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b0c81-f688-4a26-ab4a-6a4bf30c425f None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:26.690481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b0c81-f688-4a26-ab4a-6a4bf30c425f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:26.690946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b0c81-f688-4a26-ab4a-6a4bf30c425f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:26.693280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b0c81-f688-4a26-ab4a-6a4bf30c425f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:26.702797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 492/1009] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 6 14:09:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:26.711808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8c248-8b7a-4bda-baf9-84e40e24325c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:26.913209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8c248-8b7a-4bda-baf9-84e40e24325c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:26.969226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e47ef8-9023-4b96-9977-619aba00132c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:26.971060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e47ef8-9023-4b96-9977-619aba00132c None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:26.971060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e47ef8-9023-4b96-9977-619aba00132c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:26.971060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e47ef8-9023-4b96-9977-619aba00132c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:26.972630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e47ef8-9023-4b96-9977-619aba00132c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:26.974068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e47ef8-9023-4b96-9977-619aba00132c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:26.997760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 518/1010] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 6 14:09:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:09:27.887586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860fcb-eb2d-4745-9089-f1d3a1cf1db7 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:27.888123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860fcb-eb2d-4745-9089-f1d3a1cf1db7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:27.888429 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860fcb-eb2d-4745-9089-f1d3a1cf1db7 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:27.890923 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 493/1011] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:27.897102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc421a-4a64-4b25-af58-83003dbd43c1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:27.897969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc421a-4a64-4b25-af58-83003dbd43c1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:27.897969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc421a-4a64-4b25-af58-83003dbd43c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:27.918888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-11cc421a-4a64-4b25-af58-83003dbd43c1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:27.920178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-11cc421a-4a64-4b25-af58-83003dbd43c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:27.988249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:28.038711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 519/1012] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:27 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:28.063150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860fcb-eb2d-4745-9089-f1d3a1cf1db7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:28.256313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860fcb-eb2d-4745-9089-f1d3a1cf1db7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:28.325762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c6e56-919c-4fa9-b774-abdab97c3c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:28.327191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c6e56-919c-4fa9-b774-abdab97c3c5d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:28.327688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c6e56-919c-4fa9-b774-abdab97c3c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:28.328194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c6e56-919c-4fa9-b774-abdab97c3c5d None admin] PATH_INFO: `/v3/services/image` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:28.329524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c6e56-919c-4fa9-b774-abdab97c3c5d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:28.338258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c6e56-919c-4fa9-b774-abdab97c3c5d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-166c6e56-919c-4fa9-b774-abdab97c3c5d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:28.366919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:28.369155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:28.376076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 494/1013] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Jun 6 14:09:28 2022] GET /identity/v3/services/image => generated 86 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:28.384697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc421a-4a64-4b25-af58-83003dbd43c1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:28.418529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57dc67-a0f3-4d53-becc-34048a275de7 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:28.419034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57dc67-a0f3-4d53-becc-34048a275de7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:28.419531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57dc67-a0f3-4d53-becc-34048a275de7 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:28.421853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 495/1014] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:28.431720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5c895-05a1-4bc4-9203-8a50bc1e7bb1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:28.432358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5c895-05a1-4bc4-9203-8a50bc1e7bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:28.432843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5c895-05a1-4bc4-9203-8a50bc1e7bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:28.454723 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-02f5c895-05a1-4bc4-9203-8a50bc1e7bb1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:28.456049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-02f5c895-05a1-4bc4-9203-8a50bc1e7bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:28.459474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:28.576868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 496/1015] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:28 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:28.588086 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc421a-4a64-4b25-af58-83003dbd43c1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:28.607879 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5c895-05a1-4bc4-9203-8a50bc1e7bb1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:28.650493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc9aef-3aa9-4862-a8a3-47cd8866de9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:28.651509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc9aef-3aa9-4862-a8a3-47cd8866de9b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:28.651757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc9aef-3aa9-4862-a8a3-47cd8866de9b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:28.652122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc9aef-3aa9-4862-a8a3-47cd8866de9b None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:28.653085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc9aef-3aa9-4862-a8a3-47cd8866de9b None admin] RBAC: Adding query filter params (name=image) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:28.653489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc9aef-3aa9-4862-a8a3-47cd8866de9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:28.655769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc9aef-3aa9-4862-a8a3-47cd8866de9b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:28.665385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 520/1016] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:28.680476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc9aef-3aa9-4862-a8a3-47cd8866de9b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:28.765886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5c895-05a1-4bc4-9203-8a50bc1e7bb1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:28.777574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:28.831412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521cf55-e9f4-4f5e-a393-cbd4dfd8474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:28.832563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521cf55-e9f4-4f5e-a393-cbd4dfd8474c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:28.832985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521cf55-e9f4-4f5e-a393-cbd4dfd8474c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:28.833444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521cf55-e9f4-4f5e-a393-cbd4dfd8474c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:28.834319 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521cf55-e9f4-4f5e-a393-cbd4dfd8474c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:28.836427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521cf55-e9f4-4f5e-a393-cbd4dfd8474c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-4521cf55-e9f4-4f5e-a393-cbd4dfd8474c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:28.852793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:28.854985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 14:09:28.856216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:28.857021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc9aef-3aa9-4862-a8a3-47cd8866de9b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:28.857109 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 497/1017] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:28.862697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521cf55-e9f4-4f5e-a393-cbd4dfd8474c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:28.866084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:28.906853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca043865-487d-46e0-bfe4-503e74d76e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:28.907876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca043865-487d-46e0-bfe4-503e74d76e13 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:28.908161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca043865-487d-46e0-bfe4-503e74d76e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:28.908430 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca043865-487d-46e0-bfe4-503e74d76e13 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:28.909768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca043865-487d-46e0-bfe4-503e74d76e13 None admin] RBAC: Adding query filter params (type=image) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:28.910073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca043865-487d-46e0-bfe4-503e74d76e13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:28.911491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca043865-487d-46e0-bfe4-503e74d76e13 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:28.925807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 521/1018] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:28.926985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca043865-487d-46e0-bfe4-503e74d76e13 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:29.055470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521cf55-e9f4-4f5e-a393-cbd4dfd8474c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:29.128811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbe487-38ab-4078-bb0b-51f4423ba6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:29.129817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe487-38ab-4078-bb0b-51f4423ba6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:29.130276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe487-38ab-4078-bb0b-51f4423ba6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:29.130722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe487-38ab-4078-bb0b-51f4423ba6d8 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:29.131789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbe487-38ab-4078-bb0b-51f4423ba6d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:29.132288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbe487-38ab-4078-bb0b-51f4423ba6d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:29.134452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbe487-38ab-4078-bb0b-51f4423ba6d8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:29.135051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca043865-487d-46e0-bfe4-503e74d76e13 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:29.140823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 498/1019] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:09:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:29.147666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbe487-38ab-4078-bb0b-51f4423ba6d8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:29.157312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:29.214206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c0808-c170-4fea-bd8e-63f5144c9987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:29.215476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c0808-c170-4fea-bd8e-63f5144c9987 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:29.215958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c0808-c170-4fea-bd8e-63f5144c9987 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:29.216432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c0808-c170-4fea-bd8e-63f5144c9987 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:29.218067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c0808-c170-4fea-bd8e-63f5144c9987 None admin] RBAC: Adding query filter params (interface=public, service_id=da864a83ae2647ec919da897a27bc3af, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:29.218628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c0808-c170-4fea-bd8e-63f5144c9987 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:29.221369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c0808-c170-4fea-bd8e-63f5144c9987 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:29.234822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 522/1020] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 6 14:09:28 2022] GET /identity/v3/endpoints?service_id=da864a83ae2647ec919da897a27bc3af&interface=public®ion_id=RegionOne => generated 194 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:29.241674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c0808-c170-4fea-bd8e-63f5144c9987 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:29.278943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:29.333136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbe487-38ab-4078-bb0b-51f4423ba6d8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:29.381057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c871cc7-0ffe-4198-8753-c57376560e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:29.381622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871cc7-0ffe-4198-8753-c57376560e2a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:29.381937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871cc7-0ffe-4198-8753-c57376560e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:29.382213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871cc7-0ffe-4198-8753-c57376560e2a None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:29.383513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c871cc7-0ffe-4198-8753-c57376560e2a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:29.383819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c871cc7-0ffe-4198-8753-c57376560e2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:29.385672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c871cc7-0ffe-4198-8753-c57376560e2a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:29.392632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 499/1021] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 6 14:09:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:29.398471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c871cc7-0ffe-4198-8753-c57376560e2a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:29.458821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c0808-c170-4fea-bd8e-63f5144c9987 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:29.531814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273d966-135f-48e7-a9a9-8331f6bb0688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:29.533081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273d966-135f-48e7-a9a9-8331f6bb0688 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:29.535029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273d966-135f-48e7-a9a9-8331f6bb0688 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:29.535538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273d966-135f-48e7-a9a9-8331f6bb0688 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:29.539606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273d966-135f-48e7-a9a9-8331f6bb0688 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:29.544435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273d966-135f-48e7-a9a9-8331f6bb0688 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:29.562731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 523/1022] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 6 14:09:29 2022] GET /identity/v3/services => generated 1957 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:09:29.618512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c871cc7-0ffe-4198-8753-c57376560e2a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:29.680014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:29.691163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db014232-14dd-4bdf-8bce-e4210ed36535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:29.692501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db014232-14dd-4bdf-8bce-e4210ed36535 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:29.693156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db014232-14dd-4bdf-8bce-e4210ed36535 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:29.693822 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db014232-14dd-4bdf-8bce-e4210ed36535 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:29.695820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db014232-14dd-4bdf-8bce-e4210ed36535 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:29.698564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db014232-14dd-4bdf-8bce-e4210ed36535 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:29.734267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 500/1023] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 6 14:09:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:09:29.740486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5067e9-68c3-422d-a3f2-bbd693fdc6f6 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:29.741036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5067e9-68c3-422d-a3f2-bbd693fdc6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:29.741036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5067e9-68c3-422d-a3f2-bbd693fdc6f6 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:29.742920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 524/1024] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:29.748915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e12fc-5253-46ef-92ad-1f3d72384dd5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:29.749213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e12fc-5253-46ef-92ad-1f3d72384dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:29.749492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e12fc-5253-46ef-92ad-1f3d72384dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:29.769883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4b2e12fc-5253-46ef-92ad-1f3d72384dd5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:29.771170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4b2e12fc-5253-46ef-92ad-1f3d72384dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:29.873669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 501/1025] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:29.899054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5067e9-68c3-422d-a3f2-bbd693fdc6f6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:29.941693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:30.032895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5067e9-68c3-422d-a3f2-bbd693fdc6f6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:30.080395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4208d-5f20-419f-91a2-fcef6be14cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:30.081257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4208d-5f20-419f-91a2-fcef6be14cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:30.081549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4208d-5f20-419f-91a2-fcef6be14cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:30.081932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4208d-5f20-419f-91a2-fcef6be14cf2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:30.083447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4208d-5f20-419f-91a2-fcef6be14cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:30.085176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4208d-5f20-419f-91a2-fcef6be14cf2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:30.088162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 525/1026] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:30.095054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e12fc-5253-46ef-92ad-1f3d72384dd5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:30.248464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e12fc-5253-46ef-92ad-1f3d72384dd5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:30.265095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:30.293524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24582c67-2a6d-41ea-ad86-d8d7e11b539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:30.294245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24582c67-2a6d-41ea-ad86-d8d7e11b539e None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:30.294536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24582c67-2a6d-41ea-ad86-d8d7e11b539e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:30.294815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24582c67-2a6d-41ea-ad86-d8d7e11b539e None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:30.295546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24582c67-2a6d-41ea-ad86-d8d7e11b539e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:30.297027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24582c67-2a6d-41ea-ad86-d8d7e11b539e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:30.333811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 502/1027] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:09:30 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:09:32.158859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ae70e-06ab-46a7-9273-6508183d56c9 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:32.159608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ae70e-06ab-46a7-9273-6508183d56c9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:32.159608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ae70e-06ab-46a7-9273-6508183d56c9 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:32.161242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 526/1028] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:32.168719 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c75b6d-e718-43b5-aa96-de784cca5994 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:32.169288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c75b6d-e718-43b5-aa96-de784cca5994 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:32.169750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c75b6d-e718-43b5-aa96-de784cca5994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:32.194115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-11c75b6d-e718-43b5-aa96-de784cca5994 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:32.196067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-11c75b6d-e718-43b5-aa96-de784cca5994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:32.313905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 503/1029] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:32.337354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ae70e-06ab-46a7-9273-6508183d56c9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:32.496430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934bfb0-da17-43a7-8d24-416784c42b10 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:32.496611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934bfb0-da17-43a7-8d24-416784c42b10 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:32.496895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934bfb0-da17-43a7-8d24-416784c42b10 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:32.498619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 504/1030] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:32.508747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a8886-7196-4fcb-ac0b-e16c5b6cea81 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:32.509006 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a8886-7196-4fcb-ac0b-e16c5b6cea81 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:32.509439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a8886-7196-4fcb-ac0b-e16c5b6cea81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:32.514602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ae70e-06ab-46a7-9273-6508183d56c9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:32.519459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:32.527696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-981a8886-7196-4fcb-ac0b-e16c5b6cea81 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:32.528896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-981a8886-7196-4fcb-ac0b-e16c5b6cea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:32.560497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9118-da2b-4210-aebc-61a2ca4b4559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:32.561294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9118-da2b-4210-aebc-61a2ca4b4559 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:32.561582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9118-da2b-4210-aebc-61a2ca4b4559 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:32.561884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9118-da2b-4210-aebc-61a2ca4b4559 None admin] PATH_INFO: `/v3/services/image` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:32.562522 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a9118-da2b-4210-aebc-61a2ca4b4559 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:32.563913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a9118-da2b-4210-aebc-61a2ca4b4559 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-e26a9118-da2b-4210-aebc-61a2ca4b4559 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:32.577020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:32.579052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 14:09:32.581372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:32.584709 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 527/1031] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Jun 6 14:09:32 2022] GET /identity/v3/services/image => generated 86 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:32.584709 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9118-da2b-4210-aebc-61a2ca4b4559 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:32.627390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 505/1032] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:32.653311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a8886-7196-4fcb-ac0b-e16c5b6cea81 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:32.694426 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:32.721776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9118-da2b-4210-aebc-61a2ca4b4559 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:32.791485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd566cb-f553-4133-9324-37fd30b1fca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:32.792666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd566cb-f553-4133-9324-37fd30b1fca5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:32.793162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd566cb-f553-4133-9324-37fd30b1fca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:32.793651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd566cb-f553-4133-9324-37fd30b1fca5 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:32.795037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd566cb-f553-4133-9324-37fd30b1fca5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:32.795601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd566cb-f553-4133-9324-37fd30b1fca5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:32.798240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd566cb-f553-4133-9324-37fd30b1fca5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:32.806655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a8886-7196-4fcb-ac0b-e16c5b6cea81 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:32.809764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 528/1033] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:32.815416 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd566cb-f553-4133-9324-37fd30b1fca5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:32.872351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17e427-f454-42fc-a3fd-6eb3f76cd0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:32.873166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17e427-f454-42fc-a3fd-6eb3f76cd0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:32.873446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17e427-f454-42fc-a3fd-6eb3f76cd0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:32.873791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17e427-f454-42fc-a3fd-6eb3f76cd0b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:32.878000 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:32.890915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17e427-f454-42fc-a3fd-6eb3f76cd0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7b17e427-f454-42fc-a3fd-6eb3f76cd0b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:09:32.900685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:32.907217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:32.908284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 506/1034] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 6 14:09:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:32.911961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17e427-f454-42fc-a3fd-6eb3f76cd0b8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:32.993494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd566cb-f553-4133-9324-37fd30b1fca5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:33.052606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:33.061118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609e19-5809-468f-88de-8aed3d0f2cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:33.062317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609e19-5809-468f-88de-8aed3d0f2cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:33.062830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609e19-5809-468f-88de-8aed3d0f2cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:33.063308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609e19-5809-468f-88de-8aed3d0f2cc3 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:33.064627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609e19-5809-468f-88de-8aed3d0f2cc3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:33.065182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609e19-5809-468f-88de-8aed3d0f2cc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:33.067727 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609e19-5809-468f-88de-8aed3d0f2cc3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:33.076043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17e427-f454-42fc-a3fd-6eb3f76cd0b8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:33.077568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 529/1035] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:33.086766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609e19-5809-468f-88de-8aed3d0f2cc3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:33.135948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40325f-6ae5-4ba9-8551-9ceffffa7078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:33.136635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40325f-6ae5-4ba9-8551-9ceffffa7078 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:33.136951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40325f-6ae5-4ba9-8551-9ceffffa7078 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:33.137206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40325f-6ae5-4ba9-8551-9ceffffa7078 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:33.138031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40325f-6ae5-4ba9-8551-9ceffffa7078 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:33.138338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40325f-6ae5-4ba9-8551-9ceffffa7078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:33.139762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40325f-6ae5-4ba9-8551-9ceffffa7078 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:33.158810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 507/1036] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 6 14:09:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:33.161227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40325f-6ae5-4ba9-8551-9ceffffa7078 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:33.239632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:33.247344 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609e19-5809-468f-88de-8aed3d0f2cc3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:33.291327 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d503016-59cf-4ce7-a9ed-206db337bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:33.292004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d503016-59cf-4ce7-a9ed-206db337bfc2 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:33.292284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d503016-59cf-4ce7-a9ed-206db337bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:33.292611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d503016-59cf-4ce7-a9ed-206db337bfc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:33.293369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d503016-59cf-4ce7-a9ed-206db337bfc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:33.294815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d503016-59cf-4ce7-a9ed-206db337bfc2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:33.325880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 530/1037] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 6 14:09:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:09:33.358731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40325f-6ae5-4ba9-8551-9ceffffa7078 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:33.412305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fc101-cccb-4053-a1b7-3c4c307c96a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:33.414115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fc101-cccb-4053-a1b7-3c4c307c96a8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:33.415405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fc101-cccb-4053-a1b7-3c4c307c96a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:33.415916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fc101-cccb-4053-a1b7-3c4c307c96a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:33.430346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fc101-cccb-4053-a1b7-3c4c307c96a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:33.432298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fc101-cccb-4053-a1b7-3c4c307c96a8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:33.434855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8d2fc101-cccb-4053-a1b7-3c4c307c96a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:33.455952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:33.457839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:33.465564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:33.465564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:33.465564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:33.465564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:33.465564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 508/1038] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:33.467816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d503016-59cf-4ce7-a9ed-206db337bfc2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:33.612346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d503016-59cf-4ce7-a9ed-206db337bfc2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:33.681801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a748d-a049-4c22-9db8-d5f6f251fe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:33.682894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a748d-a049-4c22-9db8-d5f6f251fe8c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:33.683372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a748d-a049-4c22-9db8-d5f6f251fe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:33.683868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a748d-a049-4c22-9db8-d5f6f251fe8c None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:33.685219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a748d-a049-4c22-9db8-d5f6f251fe8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:33.685792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a748d-a049-4c22-9db8-d5f6f251fe8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:33.688360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a748d-a049-4c22-9db8-d5f6f251fe8c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:33.711168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efdb86-956d-4783-8e9e-c44ab075e8d6 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:33.711433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efdb86-956d-4783-8e9e-c44ab075e8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:33.711768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efdb86-956d-4783-8e9e-c44ab075e8d6 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:33.712162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 531/1039] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:33.713368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 509/1040] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:33.718939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08af127-21c2-4e5a-9519-02ea276c00e3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:33.719233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08af127-21c2-4e5a-9519-02ea276c00e3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:33.719512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08af127-21c2-4e5a-9519-02ea276c00e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:33.721241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efdb86-956d-4783-8e9e-c44ab075e8d6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:33.741591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c08af127-21c2-4e5a-9519-02ea276c00e3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:33.743770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c08af127-21c2-4e5a-9519-02ea276c00e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:33.755446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:33.834504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 532/1041] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:33.846978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efdb86-956d-4783-8e9e-c44ab075e8d6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:33.859993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08af127-21c2-4e5a-9519-02ea276c00e3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:33.887593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b81a9-f330-4afa-915c-8acb74e479ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:33.888305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b81a9-f330-4afa-915c-8acb74e479ab None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:33.888586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b81a9-f330-4afa-915c-8acb74e479ab None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:33.888859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b81a9-f330-4afa-915c-8acb74e479ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:33.896992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b81a9-f330-4afa-915c-8acb74e479ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:33.899030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b81a9-f330-4afa-915c-8acb74e479ab None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ba5b81a9-f330-4afa-915c-8acb74e479ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:09:33.916146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:33.917294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:33.918382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 510/1042] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 6 14:09:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:33.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b81a9-f330-4afa-915c-8acb74e479ab None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:33.987108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08af127-21c2-4e5a-9519-02ea276c00e3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:34.001111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:34.039260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440ea8-6a36-43b0-a407-2bdcefc8e897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:34.040473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440ea8-6a36-43b0-a407-2bdcefc8e897 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:34.040894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440ea8-6a36-43b0-a407-2bdcefc8e897 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:34.041456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440ea8-6a36-43b0-a407-2bdcefc8e897 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:34.043416 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c440ea8-6a36-43b0-a407-2bdcefc8e897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:34.045684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c440ea8-6a36-43b0-a407-2bdcefc8e897 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:34.049927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 533/1043] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:34.057516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440ea8-6a36-43b0-a407-2bdcefc8e897 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:34.066260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b81a9-f330-4afa-915c-8acb74e479ab None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:34.077831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:34.139082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea9917-e903-43b1-9cd8-8364218544ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:34.139998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9917-e903-43b1-9cd8-8364218544ae None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:34.140325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9917-e903-43b1-9cd8-8364218544ae None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:34.140646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9917-e903-43b1-9cd8-8364218544ae None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:34.141544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea9917-e903-43b1-9cd8-8364218544ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:34.141964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea9917-e903-43b1-9cd8-8364218544ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:34.143854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea9917-e903-43b1-9cd8-8364218544ae None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:34.167388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 511/1044] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 6 14:09:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:34.173033 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea9917-e903-43b1-9cd8-8364218544ae None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:34.211619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440ea8-6a36-43b0-a407-2bdcefc8e897 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:34.258763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da1218-2beb-4f3a-9560-5323bd804aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:34.259440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da1218-2beb-4f3a-9560-5323bd804aab None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:34.259959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da1218-2beb-4f3a-9560-5323bd804aab None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:34.259959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da1218-2beb-4f3a-9560-5323bd804aab None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:34.260694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da1218-2beb-4f3a-9560-5323bd804aab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:34.262203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da1218-2beb-4f3a-9560-5323bd804aab None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:34.286037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 534/1045] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:09:34 2022] POST /identity/v3/users => generated 285 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:09:34.333658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea9917-e903-43b1-9cd8-8364218544ae None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:34.362502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:34.407121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef34023-28ee-4583-b3b4-d7fe913412d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:34.408149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef34023-28ee-4583-b3b4-d7fe913412d7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:34.408639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef34023-28ee-4583-b3b4-d7fe913412d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:34.409127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef34023-28ee-4583-b3b4-d7fe913412d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:34.550730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef34023-28ee-4583-b3b4-d7fe913412d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:34.552282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef34023-28ee-4583-b3b4-d7fe913412d7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:34.674712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 512/1046] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:09:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 14:09:34.684054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da1218-2beb-4f3a-9560-5323bd804aab None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:34.789925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:34.837656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da1218-2beb-4f3a-9560-5323bd804aab None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:34.883612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd76d9-c245-4110-84d6-c09d716b090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:34.884460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd76d9-c245-4110-84d6-c09d716b090a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:34.884736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd76d9-c245-4110-84d6-c09d716b090a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:34.885108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd76d9-c245-4110-84d6-c09d716b090a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:34.899294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd76d9-c245-4110-84d6-c09d716b090a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:34.900560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd76d9-c245-4110-84d6-c09d716b090a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b8cd76d9-c245-4110-84d6-c09d716b090a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:34.921449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:34.923239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:34.925020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:34.925020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:34.925020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:34.925020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:34.929810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 535/1047] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:34.930035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef34023-28ee-4583-b3b4-d7fe913412d7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:34.958628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:35.122742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef34023-28ee-4583-b3b4-d7fe913412d7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:35.162100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fd026-e1e9-4cf0-b5da-5152b86b854b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:35.162771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fd026-e1e9-4cf0-b5da-5152b86b854b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:35.163052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fd026-e1e9-4cf0-b5da-5152b86b854b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:35.163330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fd026-e1e9-4cf0-b5da-5152b86b854b None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:35.164684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fd026-e1e9-4cf0-b5da-5152b86b854b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:35.164990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fd026-e1e9-4cf0-b5da-5152b86b854b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:35.166404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fd026-e1e9-4cf0-b5da-5152b86b854b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:35.180570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 513/1048] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:35.187131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd76d9-c245-4110-84d6-c09d716b090a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:35.378017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd76d9-c245-4110-84d6-c09d716b090a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:35.409483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:35.431910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652761a-d192-4106-9704-e157d8a1bb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:35.432640 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652761a-d192-4106-9704-e157d8a1bb84 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:35.432926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652761a-d192-4106-9704-e157d8a1bb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:35.433201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652761a-d192-4106-9704-e157d8a1bb84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:35.442045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652761a-d192-4106-9704-e157d8a1bb84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:35.443622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652761a-d192-4106-9704-e157d8a1bb84 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f652761a-d192-4106-9704-e157d8a1bb84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:09:35.460080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:35.462439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:35.464217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:09:35.464217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:09:35.464217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:09:35.464217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:09:35.464217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:09:35.464217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:09:35.464217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:35.464217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:35.466241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 536/1049] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:35.468215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fd026-e1e9-4cf0-b5da-5152b86b854b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:35.571367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:35.610785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fd026-e1e9-4cf0-b5da-5152b86b854b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:35.663296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c133618-4193-4df1-85ce-04e87d6bcc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:35.664945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c133618-4193-4df1-85ce-04e87d6bcc3d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:35.665339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c133618-4193-4df1-85ce-04e87d6bcc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:35.665746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c133618-4193-4df1-85ce-04e87d6bcc3d None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:35.666990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c133618-4193-4df1-85ce-04e87d6bcc3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:35.667435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c133618-4193-4df1-85ce-04e87d6bcc3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:35.669800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c133618-4193-4df1-85ce-04e87d6bcc3d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:35.697891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 514/1050] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:09:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:35.702893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652761a-d192-4106-9704-e157d8a1bb84 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:35.892798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652761a-d192-4106-9704-e157d8a1bb84 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:35.956994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009177db-940c-4ade-be69-27e8acf79cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:35.958027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009177db-940c-4ade-be69-27e8acf79cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:35.958505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009177db-940c-4ade-be69-27e8acf79cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:35.958962 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009177db-940c-4ade-be69-27e8acf79cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:35.961079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009177db-940c-4ade-be69-27e8acf79cc1 None admin] RBAC: Adding query filter params (role.id=e57912abab304c40b99b65e14077a804, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=0b82a920df4042dd97314350aed4d7a5) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:35.961884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009177db-940c-4ade-be69-27e8acf79cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:35.964540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009177db-940c-4ade-be69-27e8acf79cc1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:35.981754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 537/1051] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:09:35 2022] GET /identity/v3/role_assignments?user.id=0b82a920df4042dd97314350aed4d7a5&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=e57912abab304c40b99b65e14077a804 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:36.010261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d520cc7-1867-4b20-93b1-00e668b2c409 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:36.010722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d520cc7-1867-4b20-93b1-00e668b2c409 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:36.011013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d520cc7-1867-4b20-93b1-00e668b2c409 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:36.013276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 515/1052] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:36.023407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c90cb3-eaa4-4311-9c12-933b0ef81fa4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:36.024852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c90cb3-eaa4-4311-9c12-933b0ef81fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:36.025161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c90cb3-eaa4-4311-9c12-933b0ef81fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:36.054243 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-54c90cb3-eaa4-4311-9c12-933b0ef81fa4 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:36.057626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-54c90cb3-eaa4-4311-9c12-933b0ef81fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:36.127484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:36.206227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 538/1053] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:36.241578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d520cc7-1867-4b20-93b1-00e668b2c409 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:36.436735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d520cc7-1867-4b20-93b1-00e668b2c409 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:36.496456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:36.509764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f0a40-9117-4506-a538-8dd36e952966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:36.509764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f0a40-9117-4506-a538-8dd36e952966 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:36.510084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f0a40-9117-4506-a538-8dd36e952966 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:36.510549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f0a40-9117-4506-a538-8dd36e952966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:36.620442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f0a40-9117-4506-a538-8dd36e952966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:36.625772 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f0a40-9117-4506-a538-8dd36e952966 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:36.753200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 516/1054] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:09:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:09:36.759164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c90cb3-eaa4-4311-9c12-933b0ef81fa4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:36.806053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c9d7f-7241-4985-8dcc-38f7c2807ff6 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:36.806369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c9d7f-7241-4985-8dcc-38f7c2807ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:36.806717 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c9d7f-7241-4985-8dcc-38f7c2807ff6 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:36.808070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 517/1055] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:36.817617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dc63f-df94-471e-95e1-48c8118707db None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:36.818186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dc63f-df94-471e-95e1-48c8118707db None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:36.819054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dc63f-df94-471e-95e1-48c8118707db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:36.849857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c85dc63f-df94-471e-95e1-48c8118707db None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:36.852523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c85dc63f-df94-471e-95e1-48c8118707db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:36.869379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c90cb3-eaa4-4311-9c12-933b0ef81fa4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:36.905808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:36.914946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8768c5-af1b-4137-927c-4fb8cd0ce370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:36.916655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8768c5-af1b-4137-927c-4fb8cd0ce370 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:36.917359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8768c5-af1b-4137-927c-4fb8cd0ce370 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:36.917918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8768c5-af1b-4137-927c-4fb8cd0ce370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:36.938499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8768c5-af1b-4137-927c-4fb8cd0ce370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:36.941096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8768c5-af1b-4137-927c-4fb8cd0ce370 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:36.958519 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 518/1056] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7e8768c5-af1b-4137-927c-4fb8cd0ce370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:36.963913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:36.964953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:36.965906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:36.965906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:36.965906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:36.965906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:36.965906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 539/1057] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:36.973761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dc63f-df94-471e-95e1-48c8118707db None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:36.989595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8768c5-af1b-4137-927c-4fb8cd0ce370 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:37.033184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:37.094740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dc63f-df94-471e-95e1-48c8118707db None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:37.147330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f12c1-b3e7-4fc0-b58a-70b39bac5c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:37.149343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f12c1-b3e7-4fc0-b58a-70b39bac5c41 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:37.149826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f12c1-b3e7-4fc0-b58a-70b39bac5c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:37.150274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f12c1-b3e7-4fc0-b58a-70b39bac5c41 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:37.151527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f12c1-b3e7-4fc0-b58a-70b39bac5c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:37.152025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f12c1-b3e7-4fc0-b58a-70b39bac5c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:37.155206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f12c1-b3e7-4fc0-b58a-70b39bac5c41 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:37.173228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8768c5-af1b-4137-927c-4fb8cd0ce370 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:37.177745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 519/1058] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:37.240829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62201e-381f-4181-b413-0b0e45f67a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:37.242261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62201e-381f-4181-b413-0b0e45f67a68 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:37.242766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62201e-381f-4181-b413-0b0e45f67a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:37.243250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62201e-381f-4181-b413-0b0e45f67a68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:37.264370 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62201e-381f-4181-b413-0b0e45f67a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1c62201e-381f-4181-b413-0b0e45f67a68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:09:37.278934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:37.280687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:09:37.282263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:37.286528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 540/1059] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:09:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:37.291974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f12c1-b3e7-4fc0-b58a-70b39bac5c41 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:37.347158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:37.509083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f12c1-b3e7-4fc0-b58a-70b39bac5c41 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:37.582639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51b37b-2f8c-45f9-a5e7-c7695f5a30a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:37.583327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51b37b-2f8c-45f9-a5e7-c7695f5a30a0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:37.583629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51b37b-2f8c-45f9-a5e7-c7695f5a30a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:37.583891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51b37b-2f8c-45f9-a5e7-c7695f5a30a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:37.584638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51b37b-2f8c-45f9-a5e7-c7695f5a30a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:37.584955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51b37b-2f8c-45f9-a5e7-c7695f5a30a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:37.587484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51b37b-2f8c-45f9-a5e7-c7695f5a30a0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:37.598756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 520/1060] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:09:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:37.606022 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62201e-381f-4181-b413-0b0e45f67a68 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:37.766692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:37.789857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62201e-381f-4181-b413-0b0e45f67a68 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:37.829033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde64c3-073e-47c6-b158-6d75ab810885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:37.829844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde64c3-073e-47c6-b158-6d75ab810885 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:37.830123 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde64c3-073e-47c6-b158-6d75ab810885 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:37.830395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde64c3-073e-47c6-b158-6d75ab810885 None admin] PATH_INFO: `/v3/users/89e5846dda1c49ffa889738f22f61524` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:37.847609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde64c3-073e-47c6-b158-6d75ab810885 None admin] RBAC: Authorizing `identity:get_user(user_id=89e5846dda1c49ffa889738f22f61524)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:37.849124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde64c3-073e-47c6-b158-6d75ab810885 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:37.851437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 541/1061] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:09:37 2022] GET /identity/v3/users/89e5846dda1c49ffa889738f22f61524 => generated 285 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:37.859070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51b37b-2f8c-45f9-a5e7-c7695f5a30a0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:38.000266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:38.020616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51b37b-2f8c-45f9-a5e7-c7695f5a30a0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:38.079351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5e7f2-ca96-421f-a150-295efe6ae913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:38.080365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5e7f2-ca96-421f-a150-295efe6ae913 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:38.080868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5e7f2-ca96-421f-a150-295efe6ae913 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:38.081363 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5e7f2-ca96-421f-a150-295efe6ae913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:38.181796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5e7f2-ca96-421f-a150-295efe6ae913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:38.183289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5e7f2-ca96-421f-a150-295efe6ae913 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:38.293647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:38.321903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 521/1062] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:09:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:09:38.328325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde64c3-073e-47c6-b158-6d75ab810885 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:38.458215 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde64c3-073e-47c6-b158-6d75ab810885 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:38.498603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc430e5b-168a-4a64-935e-f106145882b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:38.499479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc430e5b-168a-4a64-935e-f106145882b4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:38.499765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc430e5b-168a-4a64-935e-f106145882b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:38.500041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc430e5b-168a-4a64-935e-f106145882b4 None admin] PATH_INFO: `/v3/projects/87fa472afae1407fac2421ce67748437` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:38.502870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc430e5b-168a-4a64-935e-f106145882b4 None admin] RBAC: Authorizing `identity:get_project(project_id=87fa472afae1407fac2421ce67748437)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:38.504934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc430e5b-168a-4a64-935e-f106145882b4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:38.506952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 542/1063] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:09:38 2022] GET /identity/v3/projects/87fa472afae1407fac2421ce67748437 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:38.513533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5e7f2-ca96-421f-a150-295efe6ae913 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:38.619295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb4d2a-6234-4004-b333-40663fe36617 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:38.619588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb4d2a-6234-4004-b333-40663fe36617 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:38.619844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb4d2a-6234-4004-b333-40663fe36617 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:38.621279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 543/1064] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:38.626479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5e7f2-ca96-421f-a150-295efe6ae913 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:38.627036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023a26b-93c5-427e-a692-b9ff1b1b04cd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:38.627330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023a26b-93c5-427e-a692-b9ff1b1b04cd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:38.627608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023a26b-93c5-427e-a692-b9ff1b1b04cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:38.645808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b023a26b-93c5-427e-a692-b9ff1b1b04cd None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:38.647053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b023a26b-93c5-427e-a692-b9ff1b1b04cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:38.651465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:38.673961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522773ff-0177-40e2-b364-26893c5bb262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:38.675014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522773ff-0177-40e2-b364-26893c5bb262 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:38.675478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522773ff-0177-40e2-b364-26893c5bb262 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:38.675942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522773ff-0177-40e2-b364-26893c5bb262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:38.677449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522773ff-0177-40e2-b364-26893c5bb262 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=87fa472afae1407fac2421ce67748437, user.id=89e5846dda1c49ffa889738f22f61524) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:38.678016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522773ff-0177-40e2-b364-26893c5bb262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:38.681339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522773ff-0177-40e2-b364-26893c5bb262 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:38.701738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 522/1065] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:09:38 2022] GET /identity/v3/role_assignments?user.id=89e5846dda1c49ffa889738f22f61524&scope.project.id=87fa472afae1407fac2421ce67748437&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:38.739488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 544/1066] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:38.766142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522773ff-0177-40e2-b364-26893c5bb262 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:38.823188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:38.882079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522773ff-0177-40e2-b364-26893c5bb262 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:38.920162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6ffa5-84c0-4760-88ef-825c0bb417c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:38.920985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6ffa5-84c0-4760-88ef-825c0bb417c6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:38.921273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6ffa5-84c0-4760-88ef-825c0bb417c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:38.921550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6ffa5-84c0-4760-88ef-825c0bb417c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:38.933696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6ffa5-84c0-4760-88ef-825c0bb417c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1fa6ffa5-84c0-4760-88ef-825c0bb417c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:09:38.942008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:38.943714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:38.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 523/1067] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 6 14:09:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:38.951661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023a26b-93c5-427e-a692-b9ff1b1b04cd None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:39.053161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023a26b-93c5-427e-a692-b9ff1b1b04cd None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:39.090271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2970ae-5beb-4eb6-9c36-4d00068a3f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:39.090956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2970ae-5beb-4eb6-9c36-4d00068a3f5c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:39.091241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2970ae-5beb-4eb6-9c36-4d00068a3f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:39.091520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2970ae-5beb-4eb6-9c36-4d00068a3f5c None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:39.092288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2970ae-5beb-4eb6-9c36-4d00068a3f5c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:39.092596 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2970ae-5beb-4eb6-9c36-4d00068a3f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:39.094017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2970ae-5beb-4eb6-9c36-4d00068a3f5c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:39.102782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 545/1068] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 6 14:09:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:39.108786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6ffa5-84c0-4760-88ef-825c0bb417c6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:39.217346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6ffa5-84c0-4760-88ef-825c0bb417c6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:39.227221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:39.256770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6667ae8-4172-423e-9378-b09c6e2cfb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:39.258281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6667ae8-4172-423e-9378-b09c6e2cfb6b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:39.258576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6667ae8-4172-423e-9378-b09c6e2cfb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:39.258854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6667ae8-4172-423e-9378-b09c6e2cfb6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:39.270346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6667ae8-4172-423e-9378-b09c6e2cfb6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:39.271819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6667ae8-4172-423e-9378-b09c6e2cfb6b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c6667ae8-4172-423e-9378-b09c6e2cfb6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:39.290574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:39.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:39.293316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:39.293316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:39.293316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:39.293316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:39.293316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 524/1069] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:39.297966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2970ae-5beb-4eb6-9c36-4d00068a3f5c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:39.300890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:39.409804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2970ae-5beb-4eb6-9c36-4d00068a3f5c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:39.446078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3f28f-0ffa-4ff3-ae16-85f0af88c938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:39.446775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3f28f-0ffa-4ff3-ae16-85f0af88c938 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:39.447062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3f28f-0ffa-4ff3-ae16-85f0af88c938 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:39.447340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3f28f-0ffa-4ff3-ae16-85f0af88c938 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:39.448097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3f28f-0ffa-4ff3-ae16-85f0af88c938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:39.448407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3f28f-0ffa-4ff3-ae16-85f0af88c938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:39.449890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed3f28f-0ffa-4ff3-ae16-85f0af88c938 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:39.464668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 546/1070] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:39.470325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6667ae8-4172-423e-9378-b09c6e2cfb6b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:39.579256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6667ae8-4172-423e-9378-b09c6e2cfb6b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:39.620201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1a99f-62fd-473b-ab8f-b8a1ffc11d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:39.621064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1a99f-62fd-473b-ab8f-b8a1ffc11d66 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:39.621393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1a99f-62fd-473b-ab8f-b8a1ffc11d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:39.621742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1a99f-62fd-473b-ab8f-b8a1ffc11d66 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:39.630419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1a99f-62fd-473b-ab8f-b8a1ffc11d66 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:39.631954 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1a99f-62fd-473b-ab8f-b8a1ffc11d66 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-3cb1a99f-62fd-473b-ab8f-b8a1ffc11d66 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:09:39.647476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:39.650062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:39.652536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 525/1071] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 6 14:09:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:39.656473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3f28f-0ffa-4ff3-ae16-85f0af88c938 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:39.736288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:39.813405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3f28f-0ffa-4ff3-ae16-85f0af88c938 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:39.880357 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00982de4-6dea-4981-ab95-3389962851e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:39.881675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00982de4-6dea-4981-ab95-3389962851e4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:39.883055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00982de4-6dea-4981-ab95-3389962851e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:39.883731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00982de4-6dea-4981-ab95-3389962851e4 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:39.885138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00982de4-6dea-4981-ab95-3389962851e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:39.885675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00982de4-6dea-4981-ab95-3389962851e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:39.888425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00982de4-6dea-4981-ab95-3389962851e4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:39.921605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 547/1072] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 6 14:09:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:39.928010 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1a99f-62fd-473b-ab8f-b8a1ffc11d66 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:39.973498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:40.072076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1a99f-62fd-473b-ab8f-b8a1ffc11d66 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:40.115125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73152a-9d9c-4fb7-bc64-be3d80ec8b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:40.116079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73152a-9d9c-4fb7-bc64-be3d80ec8b79 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:40.116450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73152a-9d9c-4fb7-bc64-be3d80ec8b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:40.116998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73152a-9d9c-4fb7-bc64-be3d80ec8b79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:40.130345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73152a-9d9c-4fb7-bc64-be3d80ec8b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:40.131955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73152a-9d9c-4fb7-bc64-be3d80ec8b79 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ec73152a-9d9c-4fb7-bc64-be3d80ec8b79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:40.152679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:40.154699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:40.156419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:40.156419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:40.156419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:40.156419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:40.156419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 526/1073] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:40.163889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00982de4-6dea-4981-ab95-3389962851e4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:40.353558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00982de4-6dea-4981-ab95-3389962851e4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:40.391323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:40.417953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d20e44-7fb1-49c5-81b9-1f16ec9c77fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:40.419088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d20e44-7fb1-49c5-81b9-1f16ec9c77fb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:40.419589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d20e44-7fb1-49c5-81b9-1f16ec9c77fb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:40.420064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d20e44-7fb1-49c5-81b9-1f16ec9c77fb None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:40.421361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d20e44-7fb1-49c5-81b9-1f16ec9c77fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:40.421935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d20e44-7fb1-49c5-81b9-1f16ec9c77fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:40.424493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d20e44-7fb1-49c5-81b9-1f16ec9c77fb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:40.451409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 548/1074] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:40.452851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73152a-9d9c-4fb7-bc64-be3d80ec8b79 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:40.582670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:40.646361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73152a-9d9c-4fb7-bc64-be3d80ec8b79 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:40.691625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb9000-c84b-4323-98ac-86c4ccd9b166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:40.692466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb9000-c84b-4323-98ac-86c4ccd9b166 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:40.692745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb9000-c84b-4323-98ac-86c4ccd9b166 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:40.693022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb9000-c84b-4323-98ac-86c4ccd9b166 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:40.701639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb9000-c84b-4323-98ac-86c4ccd9b166 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:40.703154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb9000-c84b-4323-98ac-86c4ccd9b166 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a9fb9000-c84b-4323-98ac-86c4ccd9b166 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:09:40.719683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:40.721357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:40.722831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:09:40.722831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:09:40.722831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:09:40.722831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:09:40.722831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:09:40.722831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:09:40.722831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:40.722831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:40.722831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 527/1075] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:40.726903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d20e44-7fb1-49c5-81b9-1f16ec9c77fb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:40.858007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d20e44-7fb1-49c5-81b9-1f16ec9c77fb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:40.903831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4f17-2da4-4052-8ed6-d26158d82ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:40.904401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4f17-2da4-4052-8ed6-d26158d82ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:40.904753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4f17-2da4-4052-8ed6-d26158d82ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:40.905030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4f17-2da4-4052-8ed6-d26158d82ab6 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:40.905902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4f17-2da4-4052-8ed6-d26158d82ab6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:40.906210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4f17-2da4-4052-8ed6-d26158d82ab6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:40.907691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4f17-2da4-4052-8ed6-d26158d82ab6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:40.921080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 549/1076] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:09:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:40.927924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb9000-c84b-4323-98ac-86c4ccd9b166 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:41.032829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640d5c-0406-4209-8a70-b606f7f501ed None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:41.033337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640d5c-0406-4209-8a70-b606f7f501ed None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:41.033888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640d5c-0406-4209-8a70-b606f7f501ed None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:41.036204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 550/1077] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:41.045436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb9000-c84b-4323-98ac-86c4ccd9b166 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:41.045915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b22dcc-5706-495d-861c-6b0fb30e6645 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:41.046406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b22dcc-5706-495d-861c-6b0fb30e6645 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:41.046887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b22dcc-5706-495d-861c-6b0fb30e6645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:41.073148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-83b22dcc-5706-495d-861c-6b0fb30e6645 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:41.074457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-83b22dcc-5706-495d-861c-6b0fb30e6645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:41.092468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:41.100541 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731b3c8-a103-45a6-b079-d429e06541bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:41.102549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731b3c8-a103-45a6-b079-d429e06541bb None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:41.102951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731b3c8-a103-45a6-b079-d429e06541bb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:41.103406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731b3c8-a103-45a6-b079-d429e06541bb None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/0b82a920df4042dd97314350aed4d7a5/roles/e57912abab304c40b99b65e14077a804` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:41.107029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731b3c8-a103-45a6-b079-d429e06541bb None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=0b82a920df4042dd97314350aed4d7a5, role_id=e57912abab304c40b99b65e14077a804)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:41.109593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731b3c8-a103-45a6-b079-d429e06541bb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:41.130856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 528/1078] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:09:40 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/0b82a920df4042dd97314350aed4d7a5/roles/e57912abab304c40b99b65e14077a804 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:09:41.138318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:41.186060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 551/1079] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:41.209915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731b3c8-a103-45a6-b079-d429e06541bb None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:41.342239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731b3c8-a103-45a6-b079-d429e06541bb None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:41.385601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cc3b0-c3a8-4d46-86fc-cdc61915cf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:41.386445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cc3b0-c3a8-4d46-86fc-cdc61915cf01 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:41.386722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cc3b0-c3a8-4d46-86fc-cdc61915cf01 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:41.386986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cc3b0-c3a8-4d46-86fc-cdc61915cf01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:41.399805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cc3b0-c3a8-4d46-86fc-cdc61915cf01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-147cc3b0-c3a8-4d46-86fc-cdc61915cf01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:09:41.407562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:41.409550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:41.411135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 529/1080] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:09:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:41.415410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b22dcc-5706-495d-861c-6b0fb30e6645 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:41.546229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b22dcc-5706-495d-861c-6b0fb30e6645 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:41.601639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf3a72-37e3-44df-bcc8-d463e9af3cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:41.602552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf3a72-37e3-44df-bcc8-d463e9af3cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:41.602835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf3a72-37e3-44df-bcc8-d463e9af3cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:41.603173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf3a72-37e3-44df-bcc8-d463e9af3cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:41.603948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf3a72-37e3-44df-bcc8-d463e9af3cf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:41.604252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf3a72-37e3-44df-bcc8-d463e9af3cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:41.605689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf3a72-37e3-44df-bcc8-d463e9af3cf7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:41.615469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 552/1081] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:09:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:41.622789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cc3b0-c3a8-4d46-86fc-cdc61915cf01 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:41.690367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:41.778678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cc3b0-c3a8-4d46-86fc-cdc61915cf01 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:41.837913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdbea8-0add-42f1-8eeb-31446f9d4995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:41.839347 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdbea8-0add-42f1-8eeb-31446f9d4995 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:41.839839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdbea8-0add-42f1-8eeb-31446f9d4995 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:41.840325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdbea8-0add-42f1-8eeb-31446f9d4995 None admin] PATH_INFO: `/v3/users/89e5846dda1c49ffa889738f22f61524` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:41.842418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdbea8-0add-42f1-8eeb-31446f9d4995 None admin] RBAC: Authorizing `identity:get_user(user_id=89e5846dda1c49ffa889738f22f61524)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:41.845138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdbea8-0add-42f1-8eeb-31446f9d4995 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:41.848265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 530/1082] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:09:41 2022] GET /identity/v3/users/89e5846dda1c49ffa889738f22f61524 => generated 285 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:41.857148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf3a72-37e3-44df-bcc8-d463e9af3cf7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:41.894148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:41.974667 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf3a72-37e3-44df-bcc8-d463e9af3cf7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:42.023641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f61fdc-321c-4725-875c-b94278f083bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:42.024517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f61fdc-321c-4725-875c-b94278f083bb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:42.024803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f61fdc-321c-4725-875c-b94278f083bb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:42.025087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f61fdc-321c-4725-875c-b94278f083bb None admin] PATH_INFO: `/v3/projects/87fa472afae1407fac2421ce67748437` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:42.026369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f61fdc-321c-4725-875c-b94278f083bb None admin] RBAC: Authorizing `identity:get_project(project_id=87fa472afae1407fac2421ce67748437)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:42.027855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f61fdc-321c-4725-875c-b94278f083bb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:42.029608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 553/1083] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:09:41 2022] GET /identity/v3/projects/87fa472afae1407fac2421ce67748437 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:42.036284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdbea8-0add-42f1-8eeb-31446f9d4995 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:42.172315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdbea8-0add-42f1-8eeb-31446f9d4995 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:42.215674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:42.242493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b881a-e975-416b-af96-b9bdc805ec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:42.243809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b881a-e975-416b-af96-b9bdc805ec11 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:42.244317 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b881a-e975-416b-af96-b9bdc805ec11 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:42.244819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b881a-e975-416b-af96-b9bdc805ec11 None admin] PATH_INFO: `/v3/projects/87fa472afae1407fac2421ce67748437/users/89e5846dda1c49ffa889738f22f61524/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:42.248685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b881a-e975-416b-af96-b9bdc805ec11 None admin] RBAC: Authorizing `identity:create_grant(project_id=87fa472afae1407fac2421ce67748437, user_id=89e5846dda1c49ffa889738f22f61524, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:42.251575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b881a-e975-416b-af96-b9bdc805ec11 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:42.273446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 531/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:09:42 2022] PUT /identity/v3/projects/87fa472afae1407fac2421ce67748437/users/89e5846dda1c49ffa889738f22f61524/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:09:43.558138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614672c-586e-40e4-b925-6153478adb58 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:43.558724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614672c-586e-40e4-b925-6153478adb58 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:43.559143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614672c-586e-40e4-b925-6153478adb58 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:43.561835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 554/1085] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:43.571356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086e0f7-3e44-4a76-83a4-380c1cd194b2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:43.572223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086e0f7-3e44-4a76-83a4-380c1cd194b2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:43.572444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086e0f7-3e44-4a76-83a4-380c1cd194b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:43.592854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0086e0f7-3e44-4a76-83a4-380c1cd194b2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:43.594313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0086e0f7-3e44-4a76-83a4-380c1cd194b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:43.696386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 532/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:43.721478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614672c-586e-40e4-b925-6153478adb58 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:43.875712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:43.880851 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614672c-586e-40e4-b925-6153478adb58 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:43.922045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c7680-6880-4382-a008-d211b8b29df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:43.922940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c7680-6880-4382-a008-d211b8b29df1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:43.923282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c7680-6880-4382-a008-d211b8b29df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:43.923614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c7680-6880-4382-a008-d211b8b29df1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:43.937212 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c7680-6880-4382-a008-d211b8b29df1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f99c7680-6880-4382-a008-d211b8b29df1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:09:43.946882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:43.948882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:43.949911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 555/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 6 14:09:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:43.958302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086e0f7-3e44-4a76-83a4-380c1cd194b2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:44.100188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086e0f7-3e44-4a76-83a4-380c1cd194b2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:44.114499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:44.147432 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf11567-31bf-4690-9ce3-25c71b7d530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:44.148071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf11567-31bf-4690-9ce3-25c71b7d530a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:44.148542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf11567-31bf-4690-9ce3-25c71b7d530a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:44.149027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf11567-31bf-4690-9ce3-25c71b7d530a None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:44.149905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf11567-31bf-4690-9ce3-25c71b7d530a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:44.150216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf11567-31bf-4690-9ce3-25c71b7d530a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:44.151689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf11567-31bf-4690-9ce3-25c71b7d530a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:44.161803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 533/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 6 14:09:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:44.169575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c7680-6880-4382-a008-d211b8b29df1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:44.301626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c7680-6880-4382-a008-d211b8b29df1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:44.341064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd70f7-0981-4ed7-a02b-958fb6efe561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:44.341900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd70f7-0981-4ed7-a02b-958fb6efe561 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:44.342178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd70f7-0981-4ed7-a02b-958fb6efe561 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:44.342453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd70f7-0981-4ed7-a02b-958fb6efe561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:44.354515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd70f7-0981-4ed7-a02b-958fb6efe561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:44.355997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd70f7-0981-4ed7-a02b-958fb6efe561 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-36cd70f7-0981-4ed7-a02b-958fb6efe561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:44.376511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:44.378630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:44.380831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:44.380831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:44.380831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:44.380831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:44.380831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 556/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:44.384291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf11567-31bf-4690-9ce3-25c71b7d530a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:44.513671 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf11567-31bf-4690-9ce3-25c71b7d530a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:44.563168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bd4c3-fdab-486d-8a67-c4d89fd2152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:44.564321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bd4c3-fdab-486d-8a67-c4d89fd2152f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:44.564808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bd4c3-fdab-486d-8a67-c4d89fd2152f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:44.565296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bd4c3-fdab-486d-8a67-c4d89fd2152f None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:44.566736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bd4c3-fdab-486d-8a67-c4d89fd2152f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:44.567279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bd4c3-fdab-486d-8a67-c4d89fd2152f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:44.570568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bd4c3-fdab-486d-8a67-c4d89fd2152f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:44.598777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 534/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:44.601481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd70f7-0981-4ed7-a02b-958fb6efe561 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:44.629325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:44.740068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd70f7-0981-4ed7-a02b-958fb6efe561 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:44.751025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37530ddd-ced4-4c67-afd3-733f297fb673 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:44.751491 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37530ddd-ced4-4c67-afd3-733f297fb673 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:44.751951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37530ddd-ced4-4c67-afd3-733f297fb673 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:44.754301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 535/1091] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:44.763383 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73fbc4-9ad2-422c-a4ce-340cb8e9da83 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:44.763917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73fbc4-9ad2-422c-a4ce-340cb8e9da83 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:44.764306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73fbc4-9ad2-422c-a4ce-340cb8e9da83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:44.780188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221612f0-8fb1-44ac-9b08-d9cac0c7d763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:44.781080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221612f0-8fb1-44ac-9b08-d9cac0c7d763 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:44.781420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221612f0-8fb1-44ac-9b08-d9cac0c7d763 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:44.781793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221612f0-8fb1-44ac-9b08-d9cac0c7d763 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:44.790804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221612f0-8fb1-44ac-9b08-d9cac0c7d763 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:44.792273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221612f0-8fb1-44ac-9b08-d9cac0c7d763 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:44.797983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5b73fbc4-9ad2-422c-a4ce-340cb8e9da83 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:44.799919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5b73fbc4-9ad2-422c-a4ce-340cb8e9da83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-221612f0-8fb1-44ac-9b08-d9cac0c7d763 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:09:44.808608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:44.809839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:44.812361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 557/1092] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 6 14:09:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:09:44.818207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221612f0-8fb1-44ac-9b08-d9cac0c7d763 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:44.848285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:44.900965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 536/1093] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:44.908132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:44.925566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73fbc4-9ad2-422c-a4ce-340cb8e9da83 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:44.942411 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221612f0-8fb1-44ac-9b08-d9cac0c7d763 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:44.993052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4ab9d-b6b5-4cf0-9a86-f1521d5b114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:44.993764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4ab9d-b6b5-4cf0-9a86-f1521d5b114f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:44.994045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4ab9d-b6b5-4cf0-9a86-f1521d5b114f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:44.994319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4ab9d-b6b5-4cf0-9a86-f1521d5b114f None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:44.998855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4ab9d-b6b5-4cf0-9a86-f1521d5b114f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:44.998855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4ab9d-b6b5-4cf0-9a86-f1521d5b114f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:44.998855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4ab9d-b6b5-4cf0-9a86-f1521d5b114f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:45.018758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 558/1094] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 6 14:09:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:45.026429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4ab9d-b6b5-4cf0-9a86-f1521d5b114f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:45.060402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73fbc4-9ad2-422c-a4ce-340cb8e9da83 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:45.098696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e18a-c139-4447-91a8-ca930fcd1e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:45.099520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e18a-c139-4447-91a8-ca930fcd1e2d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:45.099789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e18a-c139-4447-91a8-ca930fcd1e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:45.100061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e18a-c139-4447-91a8-ca930fcd1e2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:45.112341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917e18a-c139-4447-91a8-ca930fcd1e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2917e18a-c139-4447-91a8-ca930fcd1e2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:09:45.121959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:45.127189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 537/1095] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:09:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:45.129761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e18a-c139-4447-91a8-ca930fcd1e2d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:45.147052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4ab9d-b6b5-4cf0-9a86-f1521d5b114f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:45.157090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:45.178737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:45.188626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701f735-61b5-457c-b1aa-9b6ee1db1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:45.189411 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701f735-61b5-457c-b1aa-9b6ee1db1c34 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:45.189829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701f735-61b5-457c-b1aa-9b6ee1db1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:45.190195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701f735-61b5-457c-b1aa-9b6ee1db1c34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:45.256222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e18a-c139-4447-91a8-ca930fcd1e2d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:45.263518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701f735-61b5-457c-b1aa-9b6ee1db1c34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:45.265005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701f735-61b5-457c-b1aa-9b6ee1db1c34 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:45.317198 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caced634-fde6-4492-a4ec-982d823c225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:45.317903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caced634-fde6-4492-a4ec-982d823c225d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:45.318192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caced634-fde6-4492-a4ec-982d823c225d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:45.318470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caced634-fde6-4492-a4ec-982d823c225d None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:45.319217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caced634-fde6-4492-a4ec-982d823c225d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:45.320048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caced634-fde6-4492-a4ec-982d823c225d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:45.321622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caced634-fde6-4492-a4ec-982d823c225d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:45.331484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 538/1096] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:09:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:45.337811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caced634-fde6-4492-a4ec-982d823c225d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:45.376999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 559/1097] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:09:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 14:09:45.383957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701f735-61b5-457c-b1aa-9b6ee1db1c34 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:45.421819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:45.460018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:45.479273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caced634-fde6-4492-a4ec-982d823c225d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:45.513464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701f735-61b5-457c-b1aa-9b6ee1db1c34 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:45.517234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cb354-d091-4171-b9c0-ab84eae872fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:45.518250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cb354-d091-4171-b9c0-ab84eae872fa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:45.518425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cb354-d091-4171-b9c0-ab84eae872fa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:45.518665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cb354-d091-4171-b9c0-ab84eae872fa None admin] PATH_INFO: `/v3/users/89e5846dda1c49ffa889738f22f61524` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:45.519906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cb354-d091-4171-b9c0-ab84eae872fa None admin] RBAC: Authorizing `identity:get_user(user_id=89e5846dda1c49ffa889738f22f61524)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:45.521329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cb354-d091-4171-b9c0-ab84eae872fa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:45.523119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 539/1098] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:09:45 2022] GET /identity/v3/users/89e5846dda1c49ffa889738f22f61524 => generated 285 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:45.529978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cb354-d091-4171-b9c0-ab84eae872fa None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:45.552698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044b05b-ead5-4cb2-b965-44e701fd2759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:45.553505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044b05b-ead5-4cb2-b965-44e701fd2759 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:45.553826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044b05b-ead5-4cb2-b965-44e701fd2759 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:45.554399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044b05b-ead5-4cb2-b965-44e701fd2759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:45.565665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044b05b-ead5-4cb2-b965-44e701fd2759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:45.567172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044b05b-ead5-4cb2-b965-44e701fd2759 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2044b05b-ead5-4cb2-b965-44e701fd2759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:09:45.586408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:45.587888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:45.589609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:09:45.589609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:09:45.589609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:09:45.589609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:45.589609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 560/1099] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:45.594420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044b05b-ead5-4cb2-b965-44e701fd2759 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:45.660276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cb354-d091-4171-b9c0-ab84eae872fa None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:45.692929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:45.717742 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044b05b-ead5-4cb2-b965-44e701fd2759 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:45.722537 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe15148-d9f9-4d07-8412-86b2204a5121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:45.723310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe15148-d9f9-4d07-8412-86b2204a5121 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:45.723690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe15148-d9f9-4d07-8412-86b2204a5121 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:45.724072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe15148-d9f9-4d07-8412-86b2204a5121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:45.750208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:45.757402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112a256-63c4-4adb-bb3f-bf7aa36c7d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:45.757967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112a256-63c4-4adb-bb3f-bf7aa36c7d63 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:45.758287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112a256-63c4-4adb-bb3f-bf7aa36c7d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:45.758583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112a256-63c4-4adb-bb3f-bf7aa36c7d63 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:45.759479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112a256-63c4-4adb-bb3f-bf7aa36c7d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:45.759789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112a256-63c4-4adb-bb3f-bf7aa36c7d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:45.761193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112a256-63c4-4adb-bb3f-bf7aa36c7d63 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:45.775644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 561/1100] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:45.781542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112a256-63c4-4adb-bb3f-bf7aa36c7d63 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:45.806942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe15148-d9f9-4d07-8412-86b2204a5121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:45.808893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe15148-d9f9-4d07-8412-86b2204a5121 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:45.904309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112a256-63c4-4adb-bb3f-bf7aa36c7d63 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:45.934854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 540/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:09:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 14:09:45.941788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe15148-d9f9-4d07-8412-86b2204a5121 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:45.944657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a1553-e6e4-440a-93d1-c1190d4fee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:45.946265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a1553-e6e4-440a-93d1-c1190d4fee6c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:45.946547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a1553-e6e4-440a-93d1-c1190d4fee6c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:45.946823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a1553-e6e4-440a-93d1-c1190d4fee6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:45.954816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a1553-e6e4-440a-93d1-c1190d4fee6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:45.956600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a1553-e6e4-440a-93d1-c1190d4fee6c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-eb1a1553-e6e4-440a-93d1-c1190d4fee6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:09:45.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:09:45.975441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:09:45.978475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:09:45.978475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:09:45.978475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:09:45.978475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:09:45.978475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:09:45.978475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:09:45.978475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:09:45.978475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:09:45.978475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 562/1102] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:09:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:09:45.982632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a1553-e6e4-440a-93d1-c1190d4fee6c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:45.986350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:46.022709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:46.089651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe15148-d9f9-4d07-8412-86b2204a5121 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:46.111438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a1553-e6e4-440a-93d1-c1190d4fee6c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:46.134965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980acb2d-1708-47de-9693-da220d0015d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:46.135830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980acb2d-1708-47de-9693-da220d0015d9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:46.136112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980acb2d-1708-47de-9693-da220d0015d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:46.136384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980acb2d-1708-47de-9693-da220d0015d9 None admin] PATH_INFO: `/v3/projects/87fa472afae1407fac2421ce67748437` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:46.137613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980acb2d-1708-47de-9693-da220d0015d9 None admin] RBAC: Authorizing `identity:get_project(project_id=87fa472afae1407fac2421ce67748437)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:46.139078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980acb2d-1708-47de-9693-da220d0015d9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:46.140813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 541/1103] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:09:45 2022] GET /identity/v3/projects/87fa472afae1407fac2421ce67748437 => generated 311 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:46.147087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980acb2d-1708-47de-9693-da220d0015d9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:46.158895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637a3a0-a43f-4308-9b95-b13fcd93b7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:46.159830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637a3a0-a43f-4308-9b95-b13fcd93b7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:46.160110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637a3a0-a43f-4308-9b95-b13fcd93b7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:46.160704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637a3a0-a43f-4308-9b95-b13fcd93b7a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:46.161510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637a3a0-a43f-4308-9b95-b13fcd93b7a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:46.161847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637a3a0-a43f-4308-9b95-b13fcd93b7a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:46.164132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637a3a0-a43f-4308-9b95-b13fcd93b7a5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:46.177121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 563/1104] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:09:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:46.184179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637a3a0-a43f-4308-9b95-b13fcd93b7a5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:46.272389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:46.278177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980acb2d-1708-47de-9693-da220d0015d9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:46.300332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:46.317914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d4728-33ea-4ff4-9fb1-9b416bec96ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:46.318572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d4728-33ea-4ff4-9fb1-9b416bec96ed None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:46.318855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d4728-33ea-4ff4-9fb1-9b416bec96ed None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:46.319124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d4728-33ea-4ff4-9fb1-9b416bec96ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:46.321045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d4728-33ea-4ff4-9fb1-9b416bec96ed None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=87fa472afae1407fac2421ce67748437, user.id=89e5846dda1c49ffa889738f22f61524) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:46.321045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d4728-33ea-4ff4-9fb1-9b416bec96ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:46.322046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637a3a0-a43f-4308-9b95-b13fcd93b7a5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:46.322354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d4728-33ea-4ff4-9fb1-9b416bec96ed None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:46.328675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 542/1105] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:09:46 2022] GET /identity/v3/role_assignments?user.id=89e5846dda1c49ffa889738f22f61524&scope.project.id=87fa472afae1407fac2421ce67748437&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:46.364064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce5a79-6a1b-4d38-8d59-8451aa953f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:46.364749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce5a79-6a1b-4d38-8d59-8451aa953f4b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:46.365038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce5a79-6a1b-4d38-8d59-8451aa953f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:46.365315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce5a79-6a1b-4d38-8d59-8451aa953f4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:46.366241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce5a79-6a1b-4d38-8d59-8451aa953f4b None admin] RBAC: Adding query filter params (role.id=e57912abab304c40b99b65e14077a804, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=0b82a920df4042dd97314350aed4d7a5) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:46.366629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce5a79-6a1b-4d38-8d59-8451aa953f4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:46.368177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce5a79-6a1b-4d38-8d59-8451aa953f4b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:46.385751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 564/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:09:46 2022] GET /identity/v3/role_assignments?user.id=0b82a920df4042dd97314350aed4d7a5&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=e57912abab304c40b99b65e14077a804 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:09:48.771413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00da3d5-fb6d-40e1-9f7d-6c63b54fd876 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:48.771957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00da3d5-fb6d-40e1-9f7d-6c63b54fd876 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:48.771957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00da3d5-fb6d-40e1-9f7d-6c63b54fd876 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:48.774006 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 543/1107] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:48.780398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a84ca-0250-4e95-b330-0914fc0a1790 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:48.780950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a84ca-0250-4e95-b330-0914fc0a1790 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:48.780950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a84ca-0250-4e95-b330-0914fc0a1790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:48.799803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-be7a84ca-0250-4e95-b330-0914fc0a1790 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:48.801083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-be7a84ca-0250-4e95-b330-0914fc0a1790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:48.881670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 565/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:48.908483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00da3d5-fb6d-40e1-9f7d-6c63b54fd876 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:49.017656 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00da3d5-fb6d-40e1-9f7d-6c63b54fd876 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:49.055333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872e1eb-9f60-4967-a780-2d21d0772293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:49.056122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872e1eb-9f60-4967-a780-2d21d0772293 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:49.056442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872e1eb-9f60-4967-a780-2d21d0772293 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:49.056760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872e1eb-9f60-4967-a780-2d21d0772293 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:49.058059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872e1eb-9f60-4967-a780-2d21d0772293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:49.059563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872e1eb-9f60-4967-a780-2d21d0772293 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:49.061237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 544/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:09:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:49.067212 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a84ca-0250-4e95-b330-0914fc0a1790 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:49.131045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:49.179857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a84ca-0250-4e95-b330-0914fc0a1790 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:49.217414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866255d6-a74c-4e53-adb1-049980bf0388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:49.218119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866255d6-a74c-4e53-adb1-049980bf0388 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:49.218408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866255d6-a74c-4e53-adb1-049980bf0388 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:49.218676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866255d6-a74c-4e53-adb1-049980bf0388 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:49.219407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866255d6-a74c-4e53-adb1-049980bf0388 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:49.220871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866255d6-a74c-4e53-adb1-049980bf0388 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:49.244387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 566/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:09:49 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:09:51.678267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb92bdd-b012-4827-a973-c3d6ffc5b12d None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:51.678852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb92bdd-b012-4827-a973-c3d6ffc5b12d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:51.678852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb92bdd-b012-4827-a973-c3d6ffc5b12d None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:51.680380 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 545/1111] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:51.686094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cf13b-f516-4395-95ea-ed746c0db322 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:51.686676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cf13b-f516-4395-95ea-ed746c0db322 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:51.686676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cf13b-f516-4395-95ea-ed746c0db322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:51.705477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a18cf13b-f516-4395-95ea-ed746c0db322 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:51.706765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a18cf13b-f516-4395-95ea-ed746c0db322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:51.783471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 567/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:51.808322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb92bdd-b012-4827-a973-c3d6ffc5b12d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:51.871591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:51.926557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb92bdd-b012-4827-a973-c3d6ffc5b12d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:51.962517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cc724-f8e9-469f-9745-3b870829e66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:51.963375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cc724-f8e9-469f-9745-3b870829e66f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:51.963653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cc724-f8e9-469f-9745-3b870829e66f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:51.963922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cc724-f8e9-469f-9745-3b870829e66f None admin] PATH_INFO: `/v3/roles/49053ad44c6c4cd1b932fcdb45321ff9` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:51.965396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cc724-f8e9-469f-9745-3b870829e66f None admin] RBAC: Authorizing `identity:get_role(role_id=49053ad44c6c4cd1b932fcdb45321ff9)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:51.966805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cc724-f8e9-469f-9745-3b870829e66f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:51.968196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 546/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:09:51 2022] GET /identity/v3/roles/49053ad44c6c4cd1b932fcdb45321ff9 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:51.974199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cf13b-f516-4395-95ea-ed746c0db322 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:52.020222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:52.085184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cf13b-f516-4395-95ea-ed746c0db322 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:52.121058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25b3a4-0216-4fa9-a94f-5ef7f8332c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:52.121963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25b3a4-0216-4fa9-a94f-5ef7f8332c71 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:52.122249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25b3a4-0216-4fa9-a94f-5ef7f8332c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:52.122523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25b3a4-0216-4fa9-a94f-5ef7f8332c71 None admin] PATH_INFO: `/v3/users/7e156164c1204c62af3cbac6e76a4188` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:52.137627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25b3a4-0216-4fa9-a94f-5ef7f8332c71 None admin] RBAC: Authorizing `identity:get_user(user_id=7e156164c1204c62af3cbac6e76a4188)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:52.139150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25b3a4-0216-4fa9-a94f-5ef7f8332c71 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:52.140924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 568/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:09:51 2022] GET /identity/v3/users/7e156164c1204c62af3cbac6e76a4188 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:52.146917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cc724-f8e9-469f-9745-3b870829e66f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:52.252261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cc724-f8e9-469f-9745-3b870829e66f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:52.274969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:52.291221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74be22a-9554-4fdf-9351-56ef6f34c24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:52.291936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74be22a-9554-4fdf-9351-56ef6f34c24a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:52.292268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74be22a-9554-4fdf-9351-56ef6f34c24a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:52.292645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74be22a-9554-4fdf-9351-56ef6f34c24a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:52.363422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74be22a-9554-4fdf-9351-56ef6f34c24a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:52.364807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74be22a-9554-4fdf-9351-56ef6f34c24a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:52.475617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 547/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:09:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:09:52.483685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25b3a4-0216-4fa9-a94f-5ef7f8332c71 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:52.589012 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:52.594219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25b3a4-0216-4fa9-a94f-5ef7f8332c71 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:52.655358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dedfe7-18c5-48dc-8f13-219cb0dc6a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:52.656365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dedfe7-18c5-48dc-8f13-219cb0dc6a91 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:52.656886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dedfe7-18c5-48dc-8f13-219cb0dc6a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:52.657396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dedfe7-18c5-48dc-8f13-219cb0dc6a91 None admin] PATH_INFO: `/v3/projects/87fa472afae1407fac2421ce67748437` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:52.659551 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dedfe7-18c5-48dc-8f13-219cb0dc6a91 None admin] RBAC: Authorizing `identity:get_project(project_id=87fa472afae1407fac2421ce67748437)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:52.662289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dedfe7-18c5-48dc-8f13-219cb0dc6a91 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:52.666117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 569/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:09:52 2022] GET /identity/v3/projects/87fa472afae1407fac2421ce67748437 => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:52.673810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74be22a-9554-4fdf-9351-56ef6f34c24a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:52.699721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:52.791503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74be22a-9554-4fdf-9351-56ef6f34c24a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:52.832551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694fe08-3d56-49af-8ba6-0eb3834201d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:52.833220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694fe08-3d56-49af-8ba6-0eb3834201d5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:52.833505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694fe08-3d56-49af-8ba6-0eb3834201d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:52.833815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694fe08-3d56-49af-8ba6-0eb3834201d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:52.834672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694fe08-3d56-49af-8ba6-0eb3834201d5 None admin] RBAC: Adding query filter params (role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=87fa472afae1407fac2421ce67748437, user.id=7e156164c1204c62af3cbac6e76a4188) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:52.835498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694fe08-3d56-49af-8ba6-0eb3834201d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:52.837011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694fe08-3d56-49af-8ba6-0eb3834201d5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:52.843510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 548/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:09:52 2022] GET /identity/v3/role_assignments?user.id=7e156164c1204c62af3cbac6e76a4188&scope.project.id=87fa472afae1407fac2421ce67748437&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:55.229549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dc4eb-c18c-4a35-9ebb-c58a8e17caee None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:55.230139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dc4eb-c18c-4a35-9ebb-c58a8e17caee None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:55.230312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dc4eb-c18c-4a35-9ebb-c58a8e17caee None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:55.231707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 570/1118] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:09:55.239055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7ef67-9106-4b07-aa53-736d2925946e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:55.239827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7ef67-9106-4b07-aa53-736d2925946e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:55.239827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7ef67-9106-4b07-aa53-736d2925946e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:55.258045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-70c7ef67-9106-4b07-aa53-736d2925946e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:55.259539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-70c7ef67-9106-4b07-aa53-736d2925946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:55.328587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 549/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:09:55.351937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dc4eb-c18c-4a35-9ebb-c58a8e17caee None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:55.463925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dc4eb-c18c-4a35-9ebb-c58a8e17caee None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:55.504304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d754d1-302c-4073-b612-f4987ceddee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:55.505102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d754d1-302c-4073-b612-f4987ceddee2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:55.505383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d754d1-302c-4073-b612-f4987ceddee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:55.505657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d754d1-302c-4073-b612-f4987ceddee2 None admin] PATH_INFO: `/v3/roles/49053ad44c6c4cd1b932fcdb45321ff9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:55.507808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d754d1-302c-4073-b612-f4987ceddee2 None admin] RBAC: Authorizing `identity:get_role(role_id=49053ad44c6c4cd1b932fcdb45321ff9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:55.509228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d754d1-302c-4073-b612-f4987ceddee2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:55.510743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 571/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:09:55 2022] GET /identity/v3/roles/49053ad44c6c4cd1b932fcdb45321ff9 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:55.517139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7ef67-9106-4b07-aa53-736d2925946e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:55.534152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:55.631167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7ef67-9106-4b07-aa53-736d2925946e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:55.670337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d440d1-1506-48c8-805a-d76192477f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:55.671224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d440d1-1506-48c8-805a-d76192477f83 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:55.671505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d440d1-1506-48c8-805a-d76192477f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:55.671779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d440d1-1506-48c8-805a-d76192477f83 None admin] PATH_INFO: `/v3/users/7e156164c1204c62af3cbac6e76a4188` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:55.673018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d440d1-1506-48c8-805a-d76192477f83 None admin] RBAC: Authorizing `identity:get_user(user_id=7e156164c1204c62af3cbac6e76a4188)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:55.674571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d440d1-1506-48c8-805a-d76192477f83 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:55.676299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 550/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:09:55 2022] GET /identity/v3/users/7e156164c1204c62af3cbac6e76a4188 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:55.682665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d754d1-302c-4073-b612-f4987ceddee2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:55.720867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:55.791974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d754d1-302c-4073-b612-f4987ceddee2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:55.828516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18594430-d21e-4abe-86b9-23860ded668d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:55.829371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18594430-d21e-4abe-86b9-23860ded668d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:55.829654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18594430-d21e-4abe-86b9-23860ded668d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:55.829956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18594430-d21e-4abe-86b9-23860ded668d None admin] PATH_INFO: `/v3/projects/87fa472afae1407fac2421ce67748437` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:55.831163 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18594430-d21e-4abe-86b9-23860ded668d None admin] RBAC: Authorizing `identity:get_project(project_id=87fa472afae1407fac2421ce67748437)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:55.832613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18594430-d21e-4abe-86b9-23860ded668d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:55.834384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 572/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:09:55 2022] GET /identity/v3/projects/87fa472afae1407fac2421ce67748437 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:55.840760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d440d1-1506-48c8-805a-d76192477f83 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:55.925273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:55.945100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d440d1-1506-48c8-805a-d76192477f83 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:55.980806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d6532-4414-4b2f-8a22-7e12536d049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:55.981698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d6532-4414-4b2f-8a22-7e12536d049b None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:55.982007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d6532-4414-4b2f-8a22-7e12536d049b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:55.982275 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d6532-4414-4b2f-8a22-7e12536d049b None admin] PATH_INFO: `/v3/projects/87fa472afae1407fac2421ce67748437/users/7e156164c1204c62af3cbac6e76a4188/roles/49053ad44c6c4cd1b932fcdb45321ff9` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:55.984465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d6532-4414-4b2f-8a22-7e12536d049b None admin] RBAC: Authorizing `identity:create_grant(project_id=87fa472afae1407fac2421ce67748437, user_id=7e156164c1204c62af3cbac6e76a4188, role_id=49053ad44c6c4cd1b932fcdb45321ff9)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:55.986213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d6532-4414-4b2f-8a22-7e12536d049b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:56.000715 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 551/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:09:55 2022] PUT /identity/v3/projects/87fa472afae1407fac2421ce67748437/users/7e156164c1204c62af3cbac6e76a4188/roles/49053ad44c6c4cd1b932fcdb45321ff9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:09:58.344897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac27de-75bc-4194-957f-311f183a2b70 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:58.345585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac27de-75bc-4194-957f-311f183a2b70 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:58.345585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac27de-75bc-4194-957f-311f183a2b70 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:58.346883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 573/1124] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:09:58.353357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c118b-bfe2-45e0-b27b-34742b17a76e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:58.353951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c118b-bfe2-45e0-b27b-34742b17a76e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:58.353951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c118b-bfe2-45e0-b27b-34742b17a76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:58.373171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1d0c118b-bfe2-45e0-b27b-34742b17a76e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:09:58.374446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1d0c118b-bfe2-45e0-b27b-34742b17a76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:09:58.451644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 552/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:09:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:09:58.476451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac27de-75bc-4194-957f-311f183a2b70 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:58.603946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac27de-75bc-4194-957f-311f183a2b70 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:58.606363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:58.646901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1653230-fcfb-4b04-a981-c930a6eb9461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:58.648297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1653230-fcfb-4b04-a981-c930a6eb9461 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:58.648575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1653230-fcfb-4b04-a981-c930a6eb9461 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:58.648843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1653230-fcfb-4b04-a981-c930a6eb9461 None admin] PATH_INFO: `/v3/roles/49053ad44c6c4cd1b932fcdb45321ff9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:58.650383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1653230-fcfb-4b04-a981-c930a6eb9461 None admin] RBAC: Authorizing `identity:get_role(role_id=49053ad44c6c4cd1b932fcdb45321ff9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:58.651772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1653230-fcfb-4b04-a981-c930a6eb9461 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:58.653166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 574/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:09:58 2022] GET /identity/v3/roles/49053ad44c6c4cd1b932fcdb45321ff9 => generated 223 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:58.659086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c118b-bfe2-45e0-b27b-34742b17a76e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:58.728834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:58.763802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c118b-bfe2-45e0-b27b-34742b17a76e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:58.799950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de596850-6586-4ce3-9a44-b94f33e86d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:58.800855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de596850-6586-4ce3-9a44-b94f33e86d52 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:58.801138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de596850-6586-4ce3-9a44-b94f33e86d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:58.801414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de596850-6586-4ce3-9a44-b94f33e86d52 None admin] PATH_INFO: `/v3/users/7e156164c1204c62af3cbac6e76a4188` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:58.802635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de596850-6586-4ce3-9a44-b94f33e86d52 None admin] RBAC: Authorizing `identity:get_user(user_id=7e156164c1204c62af3cbac6e76a4188)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:58.804774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de596850-6586-4ce3-9a44-b94f33e86d52 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:58.806457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 553/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:09:58 2022] GET /identity/v3/users/7e156164c1204c62af3cbac6e76a4188 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:58.812368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1653230-fcfb-4b04-a981-c930a6eb9461 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:58.962761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1653230-fcfb-4b04-a981-c930a6eb9461 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:59.022209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29531c12-ee1a-4650-b0b3-d2112d48898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:59.024711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29531c12-ee1a-4650-b0b3-d2112d48898b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:59.025224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29531c12-ee1a-4650-b0b3-d2112d48898b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:59.025731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29531c12-ee1a-4650-b0b3-d2112d48898b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:59.096354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:59.145767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29531c12-ee1a-4650-b0b3-d2112d48898b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:59.149959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29531c12-ee1a-4650-b0b3-d2112d48898b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:59.294685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 575/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:09:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 14:09:59.300575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de596850-6586-4ce3-9a44-b94f33e86d52 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:59.409890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de596850-6586-4ce3-9a44-b94f33e86d52 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:59.446200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642920c2-83f7-4970-ae4c-b03aaf409b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:59.447070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642920c2-83f7-4970-ae4c-b03aaf409b40 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:59.447388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642920c2-83f7-4970-ae4c-b03aaf409b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:59.447665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642920c2-83f7-4970-ae4c-b03aaf409b40 None admin] PATH_INFO: `/v3/projects/87fa472afae1407fac2421ce67748437` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:59.448845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642920c2-83f7-4970-ae4c-b03aaf409b40 None admin] RBAC: Authorizing `identity:get_project(project_id=87fa472afae1407fac2421ce67748437)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:59.450320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642920c2-83f7-4970-ae4c-b03aaf409b40 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:59.452031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 554/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:09:59 2022] GET /identity/v3/projects/87fa472afae1407fac2421ce67748437 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:09:59.461684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29531c12-ee1a-4650-b0b3-d2112d48898b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:09:59.538897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:09:59.572867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29531c12-ee1a-4650-b0b3-d2112d48898b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:09:59.610184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca968b9b-ab3d-4418-a922-17fc2041d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:09:59.611095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca968b9b-ab3d-4418-a922-17fc2041d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:09:59.611377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca968b9b-ab3d-4418-a922-17fc2041d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:09:59.611649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca968b9b-ab3d-4418-a922-17fc2041d1d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:09:59.612490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca968b9b-ab3d-4418-a922-17fc2041d1d8 None admin] RBAC: Adding query filter params (role.id=49053ad44c6c4cd1b932fcdb45321ff9, scope.project.id=87fa472afae1407fac2421ce67748437, user.id=7e156164c1204c62af3cbac6e76a4188) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:09:59.612803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca968b9b-ab3d-4418-a922-17fc2041d1d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:09:59.614287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca968b9b-ab3d-4418-a922-17fc2041d1d8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:09:59.621482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 576/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:09:59 2022] GET /identity/v3/role_assignments?user.id=7e156164c1204c62af3cbac6e76a4188&scope.project.id=87fa472afae1407fac2421ce67748437&role.id=49053ad44c6c4cd1b932fcdb45321ff9 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:02.005732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b4ae-f9d0-4893-8be2-4637c6fca43d None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:02.006265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b4ae-f9d0-4893-8be2-4637c6fca43d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:02.006265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b4ae-f9d0-4893-8be2-4637c6fca43d None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:02.008448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 555/1131] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:02.014447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf13a7-f317-4c2b-865b-a99a3994c53e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:02.014962 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf13a7-f317-4c2b-865b-a99a3994c53e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:02.015056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf13a7-f317-4c2b-865b-a99a3994c53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:02.032117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4dcf13a7-f317-4c2b-865b-a99a3994c53e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:02.033341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4dcf13a7-f317-4c2b-865b-a99a3994c53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:02.105314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 577/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:10:02.130853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b4ae-f9d0-4893-8be2-4637c6fca43d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:02.142560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:02.240207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b4ae-f9d0-4893-8be2-4637c6fca43d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:02.275972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd29fbf-023d-45c2-8739-dbba5ce457f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:02.276779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd29fbf-023d-45c2-8739-dbba5ce457f7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:02.277059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd29fbf-023d-45c2-8739-dbba5ce457f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:02.277331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd29fbf-023d-45c2-8739-dbba5ce457f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:02.278559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd29fbf-023d-45c2-8739-dbba5ce457f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:02.280011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd29fbf-023d-45c2-8739-dbba5ce457f7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:02.281679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 556/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:10:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:02.290679 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf13a7-f317-4c2b-865b-a99a3994c53e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:02.370627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:02.409829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf13a7-f317-4c2b-865b-a99a3994c53e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:02.448386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74891ed-6955-4232-bed1-57e35e98df11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:02.449072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74891ed-6955-4232-bed1-57e35e98df11 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:02.449353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74891ed-6955-4232-bed1-57e35e98df11 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:02.449625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74891ed-6955-4232-bed1-57e35e98df11 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:02.450401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74891ed-6955-4232-bed1-57e35e98df11 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:02.451913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74891ed-6955-4232-bed1-57e35e98df11 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:02.481874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 578/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:10:02 2022] POST /identity/v3/projects => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:10:04.845501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa983ec3-0ee5-4555-b4de-c632209bdf60 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:04.846095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa983ec3-0ee5-4555-b4de-c632209bdf60 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:04.846095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa983ec3-0ee5-4555-b4de-c632209bdf60 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:04.847380 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 557/1135] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:04.860887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803feb0-ad5f-403c-a3a8-566fa2b34e78 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:04.861562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803feb0-ad5f-403c-a3a8-566fa2b34e78 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:04.861562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803feb0-ad5f-403c-a3a8-566fa2b34e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:04.879179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d803feb0-ad5f-403c-a3a8-566fa2b34e78 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:04.880442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d803feb0-ad5f-403c-a3a8-566fa2b34e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:04.951165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 579/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:04.976593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa983ec3-0ee5-4555-b4de-c632209bdf60 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:05.075521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:05.102961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa983ec3-0ee5-4555-b4de-c632209bdf60 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:05.139503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f7831-76e9-47f5-99cf-50963a49a514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:05.140293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f7831-76e9-47f5-99cf-50963a49a514 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:05.140576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f7831-76e9-47f5-99cf-50963a49a514 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:05.140847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f7831-76e9-47f5-99cf-50963a49a514 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:05.142042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f7831-76e9-47f5-99cf-50963a49a514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:05.143475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f7831-76e9-47f5-99cf-50963a49a514 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:05.145262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 558/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:10:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:10:05.152091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803feb0-ad5f-403c-a3a8-566fa2b34e78 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:05.203397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:05.257650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803feb0-ad5f-403c-a3a8-566fa2b34e78 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:05.293288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddaf460-9ff5-4746-905c-d4c8bbbbfc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:05.293996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaf460-9ff5-4746-905c-d4c8bbbbfc4e None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:05.294498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaf460-9ff5-4746-905c-d4c8bbbbfc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:05.294974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaf460-9ff5-4746-905c-d4c8bbbbfc4e None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:05.296309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaf460-9ff5-4746-905c-d4c8bbbbfc4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:05.297884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaf460-9ff5-4746-905c-d4c8bbbbfc4e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:05.327989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 580/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:10:05 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:10:07.749079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a0c9-d068-41f8-b9ce-22c34ea506d4 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:07.749638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a0c9-d068-41f8-b9ce-22c34ea506d4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:07.749638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a0c9-d068-41f8-b9ce-22c34ea506d4 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:07.751009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 559/1139] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:07.756348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ce06a-c0c2-46d8-8de8-1bd9c40fdea7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:07.757386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ce06a-c0c2-46d8-8de8-1bd9c40fdea7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:07.757489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ce06a-c0c2-46d8-8de8-1bd9c40fdea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:07.774169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2a5ce06a-c0c2-46d8-8de8-1bd9c40fdea7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:07.775369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2a5ce06a-c0c2-46d8-8de8-1bd9c40fdea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:07.842769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 581/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:07.867251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a0c9-d068-41f8-b9ce-22c34ea506d4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:07.978605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a0c9-d068-41f8-b9ce-22c34ea506d4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:08.013645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e37455-38e5-4892-aff4-6358e1c32162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:08.014454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e37455-38e5-4892-aff4-6358e1c32162 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:08.014734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e37455-38e5-4892-aff4-6358e1c32162 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:08.015009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e37455-38e5-4892-aff4-6358e1c32162 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:08.025934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e37455-38e5-4892-aff4-6358e1c32162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-66e37455-38e5-4892-aff4-6358e1c32162 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:10:08.033559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:10:08.034871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:10:08.035832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 560/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:10:08.041890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ce06a-c0c2-46d8-8de8-1bd9c40fdea7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:08.082625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:08.155447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ce06a-c0c2-46d8-8de8-1bd9c40fdea7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:08.189781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371fe3f-2c5a-4d3b-8109-07850117dbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:08.190451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371fe3f-2c5a-4d3b-8109-07850117dbac None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:08.190741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371fe3f-2c5a-4d3b-8109-07850117dbac None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:08.191022 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371fe3f-2c5a-4d3b-8109-07850117dbac None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:08.191780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371fe3f-2c5a-4d3b-8109-07850117dbac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:08.192087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371fe3f-2c5a-4d3b-8109-07850117dbac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:08.193475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371fe3f-2c5a-4d3b-8109-07850117dbac None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:08.202454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 582/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:10:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:08.208260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e37455-38e5-4892-aff4-6358e1c32162 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:08.213068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:08.314430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e37455-38e5-4892-aff4-6358e1c32162 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:08.349861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b06f9-3b23-4521-9e81-ed6eb704997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:08.350760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b06f9-3b23-4521-9e81-ed6eb704997b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:08.351042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b06f9-3b23-4521-9e81-ed6eb704997b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:08.351321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b06f9-3b23-4521-9e81-ed6eb704997b None admin] PATH_INFO: `/v3/users/fb0cce0355f647a98ed5d761657c0878` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:08.366576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b06f9-3b23-4521-9e81-ed6eb704997b None admin] RBAC: Authorizing `identity:get_user(user_id=fb0cce0355f647a98ed5d761657c0878)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:08.368036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b06f9-3b23-4521-9e81-ed6eb704997b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:08.369802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 561/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:10:08 2022] GET /identity/v3/users/fb0cce0355f647a98ed5d761657c0878 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:08.375929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371fe3f-2c5a-4d3b-8109-07850117dbac None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:08.474667 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:08.480880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371fe3f-2c5a-4d3b-8109-07850117dbac None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:08.524127 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17baf1c6-4832-40ab-8b3c-74a3bdf69985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:08.525256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17baf1c6-4832-40ab-8b3c-74a3bdf69985 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:08.525820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17baf1c6-4832-40ab-8b3c-74a3bdf69985 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:08.526344 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17baf1c6-4832-40ab-8b3c-74a3bdf69985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:08.607583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17baf1c6-4832-40ab-8b3c-74a3bdf69985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:08.609489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17baf1c6-4832-40ab-8b3c-74a3bdf69985 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:08.711636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:08.719042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 583/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:10:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:10:08.725005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b06f9-3b23-4521-9e81-ed6eb704997b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:08.790994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:08.831575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b06f9-3b23-4521-9e81-ed6eb704997b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:08.870961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe28b4d-07da-4f4f-8ec4-a722d90ffd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:08.871934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28b4d-07da-4f4f-8ec4-a722d90ffd0f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:08.872215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28b4d-07da-4f4f-8ec4-a722d90ffd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:08.872483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28b4d-07da-4f4f-8ec4-a722d90ffd0f None admin] PATH_INFO: `/v3/projects/6754f9f0395e4304a86a854b8deed9be` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:08.873664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe28b4d-07da-4f4f-8ec4-a722d90ffd0f None admin] RBAC: Authorizing `identity:get_project(project_id=6754f9f0395e4304a86a854b8deed9be)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:08.875138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe28b4d-07da-4f4f-8ec4-a722d90ffd0f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:08.876831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 562/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:10:08 2022] GET /identity/v3/projects/6754f9f0395e4304a86a854b8deed9be => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:08.883443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17baf1c6-4832-40ab-8b3c-74a3bdf69985 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:08.995079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17baf1c6-4832-40ab-8b3c-74a3bdf69985 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:09.030963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831929c-f8d8-4377-8a97-c7af88f763c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:09.031618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831929c-f8d8-4377-8a97-c7af88f763c7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:09.031899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831929c-f8d8-4377-8a97-c7af88f763c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:09.032174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831929c-f8d8-4377-8a97-c7af88f763c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:09.033004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831929c-f8d8-4377-8a97-c7af88f763c7 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=6754f9f0395e4304a86a854b8deed9be, user.id=fb0cce0355f647a98ed5d761657c0878) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:09.033312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831929c-f8d8-4377-8a97-c7af88f763c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:09.034801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831929c-f8d8-4377-8a97-c7af88f763c7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:09.041118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 584/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:10:08 2022] GET /identity/v3/role_assignments?user.id=fb0cce0355f647a98ed5d761657c0878&scope.project.id=6754f9f0395e4304a86a854b8deed9be&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:11.324191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665dabd-f6f4-4113-bec7-c0821a213515 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:11.324781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665dabd-f6f4-4113-bec7-c0821a213515 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:11.324781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665dabd-f6f4-4113-bec7-c0821a213515 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:11.326143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 563/1147] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:11.331709 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2e03f-0bfa-4573-b402-d8aa5e5fc6f6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:11.332144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2e03f-0bfa-4573-b402-d8aa5e5fc6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:11.332272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2e03f-0bfa-4573-b402-d8aa5e5fc6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:11.349847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a7a2e03f-0bfa-4573-b402-d8aa5e5fc6f6 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:11.351309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a7a2e03f-0bfa-4573-b402-d8aa5e5fc6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:11.409466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:11.423770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 585/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:11.449488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665dabd-f6f4-4113-bec7-c0821a213515 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:11.565351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665dabd-f6f4-4113-bec7-c0821a213515 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:11.595669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:11.602956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5eb2e-a956-4d1c-b292-c0a1df6e93c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:11.603766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5eb2e-a956-4d1c-b292-c0a1df6e93c0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:11.604081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5eb2e-a956-4d1c-b292-c0a1df6e93c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:11.604356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5eb2e-a956-4d1c-b292-c0a1df6e93c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:11.615618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5eb2e-a956-4d1c-b292-c0a1df6e93c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-bfd5eb2e-a956-4d1c-b292-c0a1df6e93c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:10:11.623652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:10:11.625046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:10:11.625972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 564/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:10:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:10:11.631493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2e03f-0bfa-4573-b402-d8aa5e5fc6f6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:11.731065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2e03f-0bfa-4573-b402-d8aa5e5fc6f6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:11.764772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882905f-9078-498d-b86a-3995e62df6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:11.765438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882905f-9078-498d-b86a-3995e62df6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:11.765736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882905f-9078-498d-b86a-3995e62df6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:11.766022 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882905f-9078-498d-b86a-3995e62df6d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:11.766771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882905f-9078-498d-b86a-3995e62df6d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:11.767078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882905f-9078-498d-b86a-3995e62df6d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:11.768463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882905f-9078-498d-b86a-3995e62df6d9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:11.777382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 586/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:10:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:11.783342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5eb2e-a956-4d1c-b292-c0a1df6e93c0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:11.890090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5eb2e-a956-4d1c-b292-c0a1df6e93c0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:11.925349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e45d08-6513-4769-9067-dd86fadc3f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:11.926233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e45d08-6513-4769-9067-dd86fadc3f34 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:11.926514 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e45d08-6513-4769-9067-dd86fadc3f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:11.926786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e45d08-6513-4769-9067-dd86fadc3f34 None admin] PATH_INFO: `/v3/users/fb0cce0355f647a98ed5d761657c0878` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:11.927952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e45d08-6513-4769-9067-dd86fadc3f34 None admin] RBAC: Authorizing `identity:get_user(user_id=fb0cce0355f647a98ed5d761657c0878)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:11.929912 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e45d08-6513-4769-9067-dd86fadc3f34 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:11.931535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 565/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:10:11 2022] GET /identity/v3/users/fb0cce0355f647a98ed5d761657c0878 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:11.937287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882905f-9078-498d-b86a-3995e62df6d9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:11.997097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:12.049062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882905f-9078-498d-b86a-3995e62df6d9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:12.085944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143ad49-47ad-4f9d-ab3e-7a07152291aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:12.086824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143ad49-47ad-4f9d-ab3e-7a07152291aa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:12.087110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143ad49-47ad-4f9d-ab3e-7a07152291aa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:12.087384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143ad49-47ad-4f9d-ab3e-7a07152291aa None admin] PATH_INFO: `/v3/projects/6754f9f0395e4304a86a854b8deed9be` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:12.088621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143ad49-47ad-4f9d-ab3e-7a07152291aa None admin] RBAC: Authorizing `identity:get_project(project_id=6754f9f0395e4304a86a854b8deed9be)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:12.090174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143ad49-47ad-4f9d-ab3e-7a07152291aa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:12.091909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 587/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:10:11 2022] GET /identity/v3/projects/6754f9f0395e4304a86a854b8deed9be => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:12.097945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e45d08-6513-4769-9067-dd86fadc3f34 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:12.160344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:12.203021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e45d08-6513-4769-9067-dd86fadc3f34 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:12.237990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a691b1-7f08-4983-85e6-9a252e124729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:12.238899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a691b1-7f08-4983-85e6-9a252e124729 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:12.239179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a691b1-7f08-4983-85e6-9a252e124729 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:12.239452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a691b1-7f08-4983-85e6-9a252e124729 None admin] PATH_INFO: `/v3/projects/6754f9f0395e4304a86a854b8deed9be/users/fb0cce0355f647a98ed5d761657c0878/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:12.242310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a691b1-7f08-4983-85e6-9a252e124729 None admin] RBAC: Authorizing `identity:create_grant(project_id=6754f9f0395e4304a86a854b8deed9be, user_id=fb0cce0355f647a98ed5d761657c0878, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:12.243683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a691b1-7f08-4983-85e6-9a252e124729 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:12.256894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 566/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:10:12 2022] PUT /identity/v3/projects/6754f9f0395e4304a86a854b8deed9be/users/fb0cce0355f647a98ed5d761657c0878/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:10:14.501167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa7fc8-462d-44b4-b61f-5300979bdf7b None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:14.501734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa7fc8-462d-44b4-b61f-5300979bdf7b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:14.501734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa7fc8-462d-44b4-b61f-5300979bdf7b None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:14.503142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 588/1154] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:14.509590 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb1b82-623c-4284-9412-f0979f8c8500 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:14.510102 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb1b82-623c-4284-9412-f0979f8c8500 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:14.510214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb1b82-623c-4284-9412-f0979f8c8500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:14.527298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4dbb1b82-623c-4284-9412-f0979f8c8500 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:14.528488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4dbb1b82-623c-4284-9412-f0979f8c8500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:14.596864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 567/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:10:14.621700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa7fc8-462d-44b4-b61f-5300979bdf7b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:14.723793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa7fc8-462d-44b4-b61f-5300979bdf7b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:14.744847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:14.760592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63675c8-8ee0-4e94-874c-a60189bdd28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:14.761319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63675c8-8ee0-4e94-874c-a60189bdd28a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:14.761609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63675c8-8ee0-4e94-874c-a60189bdd28a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:14.761919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63675c8-8ee0-4e94-874c-a60189bdd28a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:14.778860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63675c8-8ee0-4e94-874c-a60189bdd28a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f63675c8-8ee0-4e94-874c-a60189bdd28a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:10:14.787540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:10:14.788699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:10:14.789816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 589/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:10:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:10:14.795488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb1b82-623c-4284-9412-f0979f8c8500 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:14.843247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:14.898353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb1b82-623c-4284-9412-f0979f8c8500 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:14.933604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff59fc-91e2-4cfe-9de1-b6cd2be9554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:14.934314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff59fc-91e2-4cfe-9de1-b6cd2be9554b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:14.934621 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff59fc-91e2-4cfe-9de1-b6cd2be9554b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:14.934921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff59fc-91e2-4cfe-9de1-b6cd2be9554b None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:14.935697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff59fc-91e2-4cfe-9de1-b6cd2be9554b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:14.936001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff59fc-91e2-4cfe-9de1-b6cd2be9554b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:14.937375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff59fc-91e2-4cfe-9de1-b6cd2be9554b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:14.946941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 568/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:10:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:14.952826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63675c8-8ee0-4e94-874c-a60189bdd28a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:15.053454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63675c8-8ee0-4e94-874c-a60189bdd28a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:15.092441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71064f46-5926-48a0-a46a-bcfcc0fc5e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:15.093315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71064f46-5926-48a0-a46a-bcfcc0fc5e31 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:15.093599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71064f46-5926-48a0-a46a-bcfcc0fc5e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:15.093905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71064f46-5926-48a0-a46a-bcfcc0fc5e31 None admin] PATH_INFO: `/v3/users/fb0cce0355f647a98ed5d761657c0878` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:15.095114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71064f46-5926-48a0-a46a-bcfcc0fc5e31 None admin] RBAC: Authorizing `identity:get_user(user_id=fb0cce0355f647a98ed5d761657c0878)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:15.096567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71064f46-5926-48a0-a46a-bcfcc0fc5e31 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:15.098233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 590/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:10:14 2022] GET /identity/v3/users/fb0cce0355f647a98ed5d761657c0878 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:15.104325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff59fc-91e2-4cfe-9de1-b6cd2be9554b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:15.205310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff59fc-91e2-4cfe-9de1-b6cd2be9554b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:15.216553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:15.244681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebb2f0-587c-472d-b3ac-da8d01f7bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:15.245336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebb2f0-587c-472d-b3ac-da8d01f7bb1a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:15.245620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebb2f0-587c-472d-b3ac-da8d01f7bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:15.245921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebb2f0-587c-472d-b3ac-da8d01f7bb1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:15.313083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebb2f0-587c-472d-b3ac-da8d01f7bb1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:15.314468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebb2f0-587c-472d-b3ac-da8d01f7bb1a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:15.438549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 569/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:10:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:10:15.445452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71064f46-5926-48a0-a46a-bcfcc0fc5e31 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:15.502160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:15.559084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71064f46-5926-48a0-a46a-bcfcc0fc5e31 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:15.598190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963866d4-ce72-4f91-bfd5-bd9b5cc4756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:15.598970 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963866d4-ce72-4f91-bfd5-bd9b5cc4756d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:15.599258 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963866d4-ce72-4f91-bfd5-bd9b5cc4756d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:15.599535 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963866d4-ce72-4f91-bfd5-bd9b5cc4756d None admin] PATH_INFO: `/v3/projects/6754f9f0395e4304a86a854b8deed9be` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:15.601269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963866d4-ce72-4f91-bfd5-bd9b5cc4756d None admin] RBAC: Authorizing `identity:get_project(project_id=6754f9f0395e4304a86a854b8deed9be)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:15.602759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963866d4-ce72-4f91-bfd5-bd9b5cc4756d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:15.604466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 591/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:10:15 2022] GET /identity/v3/projects/6754f9f0395e4304a86a854b8deed9be => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:15.610883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebb2f0-587c-472d-b3ac-da8d01f7bb1a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:15.622536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:15.723836 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebb2f0-587c-472d-b3ac-da8d01f7bb1a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:15.758710 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f62fd8-b4a9-4abb-b910-ab5a8244b2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:15.759355 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62fd8-b4a9-4abb-b910-ab5a8244b2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:15.759634 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62fd8-b4a9-4abb-b910-ab5a8244b2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:15.759907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62fd8-b4a9-4abb-b910-ab5a8244b2d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:15.760729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62fd8-b4a9-4abb-b910-ab5a8244b2d0 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=6754f9f0395e4304a86a854b8deed9be, user.id=fb0cce0355f647a98ed5d761657c0878) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:15.761036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62fd8-b4a9-4abb-b910-ab5a8244b2d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:15.762477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62fd8-b4a9-4abb-b910-ab5a8244b2d0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:15.768325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 570/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:10:15 2022] GET /identity/v3/role_assignments?user.id=fb0cce0355f647a98ed5d761657c0878&scope.project.id=6754f9f0395e4304a86a854b8deed9be&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:18.059359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3d239-de2f-488b-bfca-425baefe0781 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:18.059899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3d239-de2f-488b-bfca-425baefe0781 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:18.059899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3d239-de2f-488b-bfca-425baefe0781 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:18.061262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 592/1162] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:18.066604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4af041-87d5-4eae-9522-57d7778192c9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:18.067017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4af041-87d5-4eae-9522-57d7778192c9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:18.067170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4af041-87d5-4eae-9522-57d7778192c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:18.084394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5c4af041-87d5-4eae-9522-57d7778192c9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:18.086157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5c4af041-87d5-4eae-9522-57d7778192c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:18.152691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 571/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:18.174639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3d239-de2f-488b-bfca-425baefe0781 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:18.278900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3d239-de2f-488b-bfca-425baefe0781 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:18.295533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:18.314164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54acc39-2114-4c20-abe9-e587ec3ffebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:18.314737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54acc39-2114-4c20-abe9-e587ec3ffebd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:18.315020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54acc39-2114-4c20-abe9-e587ec3ffebd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:18.315296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54acc39-2114-4c20-abe9-e587ec3ffebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:18.385820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54acc39-2114-4c20-abe9-e587ec3ffebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:18.387353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54acc39-2114-4c20-abe9-e587ec3ffebd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:18.493217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 593/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:10:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:10:18.499578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4af041-87d5-4eae-9522-57d7778192c9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:18.503528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:18.600596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4af041-87d5-4eae-9522-57d7778192c9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:18.635161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f542d-8cd4-47b2-a946-9317290b65cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:18.635964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f542d-8cd4-47b2-a946-9317290b65cc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:18.636245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f542d-8cd4-47b2-a946-9317290b65cc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:18.636520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f542d-8cd4-47b2-a946-9317290b65cc None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:18.643189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f542d-8cd4-47b2-a946-9317290b65cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:18.644626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f542d-8cd4-47b2-a946-9317290b65cc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e03f542d-8cd4-47b2-a946-9317290b65cc None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:10:18.658662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:10:18.659970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:10:18.661142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 572/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:10:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:10:18.666228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54acc39-2114-4c20-abe9-e587ec3ffebd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:18.683651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:18.765404 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54acc39-2114-4c20-abe9-e587ec3ffebd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:18.799565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a6315-3da8-434a-8a95-26ca0186a531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:18.800228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a6315-3da8-434a-8a95-26ca0186a531 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:18.800515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a6315-3da8-434a-8a95-26ca0186a531 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:18.800796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a6315-3da8-434a-8a95-26ca0186a531 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:18.801516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a6315-3da8-434a-8a95-26ca0186a531 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:18.801857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a6315-3da8-434a-8a95-26ca0186a531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:18.803901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a6315-3da8-434a-8a95-26ca0186a531 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:18.811134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 594/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 6 14:10:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:21.086402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106bb80-1df9-4062-997e-c801572acd2c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:21.086966 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106bb80-1df9-4062-997e-c801572acd2c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:21.086966 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106bb80-1df9-4062-997e-c801572acd2c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:21.088300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 573/1167] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:21.093601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea60f7-2bee-4ccb-b0b7-610480c7e6fc None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:21.094026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea60f7-2bee-4ccb-b0b7-610480c7e6fc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:21.094201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea60f7-2bee-4ccb-b0b7-610480c7e6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:21.111158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8eea60f7-2bee-4ccb-b0b7-610480c7e6fc None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:21.112330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8eea60f7-2bee-4ccb-b0b7-610480c7e6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:21.180874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 595/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:21.204501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106bb80-1df9-4062-997e-c801572acd2c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:21.268527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:21.309888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106bb80-1df9-4062-997e-c801572acd2c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:21.345063 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c58824-2b7b-4b43-92c1-b480f88cc7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:21.345632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c58824-2b7b-4b43-92c1-b480f88cc7ae None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:21.345941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c58824-2b7b-4b43-92c1-b480f88cc7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:21.346216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c58824-2b7b-4b43-92c1-b480f88cc7ae None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:21.346950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c58824-2b7b-4b43-92c1-b480f88cc7ae None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:21.348320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c58824-2b7b-4b43-92c1-b480f88cc7ae None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:21.363097 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 574/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Mon Jun 6 14:10:21 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:10:23.731775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd6f49-30dd-4477-a902-3938d8afd581 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:23.732325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd6f49-30dd-4477-a902-3938d8afd581 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:23.732325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd6f49-30dd-4477-a902-3938d8afd581 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:23.733685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 596/1170] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:23.739002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a40850-1c3a-44fa-a60d-9bd55f35f4b1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:23.739415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a40850-1c3a-44fa-a60d-9bd55f35f4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:23.739556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a40850-1c3a-44fa-a60d-9bd55f35f4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:23.755946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c5a40850-1c3a-44fa-a60d-9bd55f35f4b1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:23.757127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c5a40850-1c3a-44fa-a60d-9bd55f35f4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:23.822542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 575/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:23.846389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd6f49-30dd-4477-a902-3938d8afd581 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:23.859187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:23.948503 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd6f49-30dd-4477-a902-3938d8afd581 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:23.983056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55778429-843d-44cc-a5e3-df9df873971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:23.983910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55778429-843d-44cc-a5e3-df9df873971e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:23.984189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55778429-843d-44cc-a5e3-df9df873971e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:23.984462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55778429-843d-44cc-a5e3-df9df873971e None admin] PATH_INFO: `/v3/domains/f4e07258e4b346588090d195ad579326` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:23.995073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55778429-843d-44cc-a5e3-df9df873971e None admin] RBAC: Authorizing `identity:get_domain(domain_id=f4e07258e4b346588090d195ad579326)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:23.996517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55778429-843d-44cc-a5e3-df9df873971e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:23.998253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 597/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Jun 6 14:10:23 2022] GET /identity/v3/domains/f4e07258e4b346588090d195ad579326 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:24.004136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a40850-1c3a-44fa-a60d-9bd55f35f4b1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:24.061252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:24.106919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a40850-1c3a-44fa-a60d-9bd55f35f4b1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:24.149215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9868aec-2515-4a79-b397-aacb3f84f3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:24.149910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9868aec-2515-4a79-b397-aacb3f84f3bf None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:24.150199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9868aec-2515-4a79-b397-aacb3f84f3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:24.150481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9868aec-2515-4a79-b397-aacb3f84f3bf None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:24.151189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9868aec-2515-4a79-b397-aacb3f84f3bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:24.152641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9868aec-2515-4a79-b397-aacb3f84f3bf None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:24.177640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 576/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:10:24 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:10:26.478453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b931057-0799-4caf-be2b-4271e5f09cde None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:26.479002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b931057-0799-4caf-be2b-4271e5f09cde None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:26.479002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b931057-0799-4caf-be2b-4271e5f09cde None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:26.480390 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 598/1174] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:26.486408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d4cef-9251-4b7f-9b8a-fc3da0556baa None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:26.486876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d4cef-9251-4b7f-9b8a-fc3da0556baa None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:26.486967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d4cef-9251-4b7f-9b8a-fc3da0556baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:26.504006 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9f3d4cef-9251-4b7f-9b8a-fc3da0556baa None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:26.505180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9f3d4cef-9251-4b7f-9b8a-fc3da0556baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:26.578730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 577/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:26.604470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b931057-0799-4caf-be2b-4271e5f09cde None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:26.674079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:26.705690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b931057-0799-4caf-be2b-4271e5f09cde None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:26.739882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437471ca-a350-411a-93e2-b9af49c65ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:26.740695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437471ca-a350-411a-93e2-b9af49c65ebc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:26.740976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437471ca-a350-411a-93e2-b9af49c65ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:26.741252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437471ca-a350-411a-93e2-b9af49c65ebc None admin] PATH_INFO: `/v3/domains/f4e07258e4b346588090d195ad579326` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:26.742469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437471ca-a350-411a-93e2-b9af49c65ebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=f4e07258e4b346588090d195ad579326)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:26.743913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437471ca-a350-411a-93e2-b9af49c65ebc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:26.745539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 599/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Jun 6 14:10:26 2022] GET /identity/v3/domains/f4e07258e4b346588090d195ad579326 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:26.751487 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d4cef-9251-4b7f-9b8a-fc3da0556baa None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:26.785449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:26.851860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d4cef-9251-4b7f-9b8a-fc3da0556baa None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:26.885855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71246f-8975-4b8a-ba92-d5a2316e5af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:26.886529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71246f-8975-4b8a-ba92-d5a2316e5af1 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:26.886811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71246f-8975-4b8a-ba92-d5a2316e5af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:26.887087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71246f-8975-4b8a-ba92-d5a2316e5af1 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:26.887809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71246f-8975-4b8a-ba92-d5a2316e5af1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:26.889225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71246f-8975-4b8a-ba92-d5a2316e5af1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:26.909175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 578/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:10:26 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:10:29.380790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75db204-3fdc-4ecf-b300-ad71830e942f None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:29.381892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75db204-3fdc-4ecf-b300-ad71830e942f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:29.381892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75db204-3fdc-4ecf-b300-ad71830e942f None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:29.383284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 600/1178] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:29.388566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302782b-b23b-4de2-9847-127acf954cdf None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:29.389017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302782b-b23b-4de2-9847-127acf954cdf None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:29.389112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302782b-b23b-4de2-9847-127acf954cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:29.406126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4302782b-b23b-4de2-9847-127acf954cdf None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:29.407299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4302782b-b23b-4de2-9847-127acf954cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:29.474019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 579/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:29.516376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75db204-3fdc-4ecf-b300-ad71830e942f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:29.626687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75db204-3fdc-4ecf-b300-ad71830e942f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:29.662011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfab49-bfbc-41e9-be03-ac2d45441e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:29.662788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfab49-bfbc-41e9-be03-ac2d45441e18 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:29.663070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfab49-bfbc-41e9-be03-ac2d45441e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:29.663351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfab49-bfbc-41e9-be03-ac2d45441e18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:29.670707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:29.675064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfab49-bfbc-41e9-be03-ac2d45441e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-04dfab49-bfbc-41e9-be03-ac2d45441e18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:10:29.683495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:10:29.684714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:10:29.685823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 601/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:10:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:10:29.690875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302782b-b23b-4de2-9847-127acf954cdf None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:29.717594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:29.798943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302782b-b23b-4de2-9847-127acf954cdf None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:29.834130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c246b-aba1-4d94-91c5-f479a31122c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:29.834975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c246b-aba1-4d94-91c5-f479a31122c8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:29.835256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c246b-aba1-4d94-91c5-f479a31122c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:29.835529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c246b-aba1-4d94-91c5-f479a31122c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:29.836271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c246b-aba1-4d94-91c5-f479a31122c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:29.836573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c246b-aba1-4d94-91c5-f479a31122c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:29.837964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c246b-aba1-4d94-91c5-f479a31122c8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:29.846339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 580/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:10:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:10:29.852456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfab49-bfbc-41e9-be03-ac2d45441e18 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:29.953142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfab49-bfbc-41e9-be03-ac2d45441e18 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:29.986482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fa4b4-253e-4d44-b759-b3d6e1fc9e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:29.987351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fa4b4-253e-4d44-b759-b3d6e1fc9e75 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:29.987634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fa4b4-253e-4d44-b759-b3d6e1fc9e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:29.987903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fa4b4-253e-4d44-b759-b3d6e1fc9e75 None admin] PATH_INFO: `/v3/users/ad4b191c4db141418794e72abc50248d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:30.001472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178fa4b4-253e-4d44-b759-b3d6e1fc9e75 None admin] RBAC: Authorizing `identity:get_user(user_id=ad4b191c4db141418794e72abc50248d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:30.002976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178fa4b4-253e-4d44-b759-b3d6e1fc9e75 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:30.004657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 602/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:10:29 2022] GET /identity/v3/users/ad4b191c4db141418794e72abc50248d => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:30.010625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c246b-aba1-4d94-91c5-f479a31122c8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:30.096288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:30.113471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c246b-aba1-4d94-91c5-f479a31122c8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:30.152826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c40d58-7a6b-40c1-9736-b86fd6849ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:30.153471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40d58-7a6b-40c1-9736-b86fd6849ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:30.153769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40d58-7a6b-40c1-9736-b86fd6849ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:30.154045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40d58-7a6b-40c1-9736-b86fd6849ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:30.222919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c40d58-7a6b-40c1-9736-b86fd6849ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:30.224274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c40d58-7a6b-40c1-9736-b86fd6849ff6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:30.307294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:30.323920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 581/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:10:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:10:30.329920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fa4b4-253e-4d44-b759-b3d6e1fc9e75 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:30.386567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:30.431635 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fa4b4-253e-4d44-b759-b3d6e1fc9e75 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:30.465631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b7a83-f036-4f42-95a2-88e6835176e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:30.466515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b7a83-f036-4f42-95a2-88e6835176e0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:30.466794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b7a83-f036-4f42-95a2-88e6835176e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:30.467064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b7a83-f036-4f42-95a2-88e6835176e0 None admin] PATH_INFO: `/v3/projects/e6dbb97bb0da4fc5b0c8e5ab55b9e021` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:30.468231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b7a83-f036-4f42-95a2-88e6835176e0 None admin] RBAC: Authorizing `identity:get_project(project_id=e6dbb97bb0da4fc5b0c8e5ab55b9e021)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:30.469664 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b7a83-f036-4f42-95a2-88e6835176e0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:30.471381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 603/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:10:30 2022] GET /identity/v3/projects/e6dbb97bb0da4fc5b0c8e5ab55b9e021 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:30.478332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c40d58-7a6b-40c1-9736-b86fd6849ff6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:30.579747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c40d58-7a6b-40c1-9736-b86fd6849ff6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:30.614164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d597a04-742d-43a8-af8b-3e216ed89cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:30.614808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d597a04-742d-43a8-af8b-3e216ed89cfe None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:30.615116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d597a04-742d-43a8-af8b-3e216ed89cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:30.615549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d597a04-742d-43a8-af8b-3e216ed89cfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:30.616372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d597a04-742d-43a8-af8b-3e216ed89cfe None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=e6dbb97bb0da4fc5b0c8e5ab55b9e021, user.id=ad4b191c4db141418794e72abc50248d) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:30.616684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d597a04-742d-43a8-af8b-3e216ed89cfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:30.618136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d597a04-742d-43a8-af8b-3e216ed89cfe None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:30.623740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 582/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:10:30 2022] GET /identity/v3/role_assignments?user.id=ad4b191c4db141418794e72abc50248d&scope.project.id=e6dbb97bb0da4fc5b0c8e5ab55b9e021&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:32.881057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3cac-4059-465b-a587-c89a5111a73c None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:32.881578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3cac-4059-465b-a587-c89a5111a73c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:32.881578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3cac-4059-465b-a587-c89a5111a73c None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:32.884623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 604/1186] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:32.890723 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af63dd-3423-416f-9857-a87b471a74f6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:32.891211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af63dd-3423-416f-9857-a87b471a74f6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:32.891305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af63dd-3423-416f-9857-a87b471a74f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:32.908617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-29af63dd-3423-416f-9857-a87b471a74f6 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:32.909930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-29af63dd-3423-416f-9857-a87b471a74f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:32.956800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:32.980267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 583/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:33.003336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a3cac-4059-465b-a587-c89a5111a73c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:33.105587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a3cac-4059-465b-a587-c89a5111a73c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:33.131280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:33.142185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d8123-eb96-447d-b6bf-d953ae10dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:33.142976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d8123-eb96-447d-b6bf-d953ae10dbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:33.143259 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d8123-eb96-447d-b6bf-d953ae10dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:33.143532 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d8123-eb96-447d-b6bf-d953ae10dbe2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:33.153646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d8123-eb96-447d-b6bf-d953ae10dbe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-009d8123-eb96-447d-b6bf-d953ae10dbe2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:10:33.161122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:10:33.162829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:10:33.163704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 605/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:10:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:10:33.169000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af63dd-3423-416f-9857-a87b471a74f6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:33.283751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af63dd-3423-416f-9857-a87b471a74f6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:33.318905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdcf5e-fef1-41e5-a3c0-0d6cc6a5a038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:33.319587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdcf5e-fef1-41e5-a3c0-0d6cc6a5a038 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:33.319866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdcf5e-fef1-41e5-a3c0-0d6cc6a5a038 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:33.320138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdcf5e-fef1-41e5-a3c0-0d6cc6a5a038 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:33.320868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdcf5e-fef1-41e5-a3c0-0d6cc6a5a038 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:33.321171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdcf5e-fef1-41e5-a3c0-0d6cc6a5a038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:33.322557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdcf5e-fef1-41e5-a3c0-0d6cc6a5a038 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:33.330715 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 584/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:10:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:33.337291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d8123-eb96-447d-b6bf-d953ae10dbe2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:33.446069 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d8123-eb96-447d-b6bf-d953ae10dbe2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:33.479475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80197603-f028-48c4-95a8-d1eea2c39e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:33.480340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80197603-f028-48c4-95a8-d1eea2c39e00 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:33.480625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80197603-f028-48c4-95a8-d1eea2c39e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:33.480900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80197603-f028-48c4-95a8-d1eea2c39e00 None admin] PATH_INFO: `/v3/users/ad4b191c4db141418794e72abc50248d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:33.482105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80197603-f028-48c4-95a8-d1eea2c39e00 None admin] RBAC: Authorizing `identity:get_user(user_id=ad4b191c4db141418794e72abc50248d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:33.483570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80197603-f028-48c4-95a8-d1eea2c39e00 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:33.485205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 606/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:10:33 2022] GET /identity/v3/users/ad4b191c4db141418794e72abc50248d => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:33.491401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdcf5e-fef1-41e5-a3c0-0d6cc6a5a038 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:33.555867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:33.614467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdcf5e-fef1-41e5-a3c0-0d6cc6a5a038 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:33.648336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89ef83-0b1f-43ac-9204-f360da87852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:33.649185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89ef83-0b1f-43ac-9204-f360da87852a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:33.649469 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89ef83-0b1f-43ac-9204-f360da87852a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:33.649772 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89ef83-0b1f-43ac-9204-f360da87852a None admin] PATH_INFO: `/v3/projects/e6dbb97bb0da4fc5b0c8e5ab55b9e021` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:33.650939 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89ef83-0b1f-43ac-9204-f360da87852a None admin] RBAC: Authorizing `identity:get_project(project_id=e6dbb97bb0da4fc5b0c8e5ab55b9e021)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:33.652383 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89ef83-0b1f-43ac-9204-f360da87852a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:33.654116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 585/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:10:33 2022] GET /identity/v3/projects/e6dbb97bb0da4fc5b0c8e5ab55b9e021 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:10:33.660164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80197603-f028-48c4-95a8-d1eea2c39e00 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:33.715073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:33.762814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80197603-f028-48c4-95a8-d1eea2c39e00 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:33.796963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54597f-ac85-4a23-92e8-651889613225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:33.797874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54597f-ac85-4a23-92e8-651889613225 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:33.798153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54597f-ac85-4a23-92e8-651889613225 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:33.798423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54597f-ac85-4a23-92e8-651889613225 None admin] PATH_INFO: `/v3/projects/e6dbb97bb0da4fc5b0c8e5ab55b9e021/users/ad4b191c4db141418794e72abc50248d/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:33.800367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54597f-ac85-4a23-92e8-651889613225 None admin] RBAC: Authorizing `identity:create_grant(project_id=e6dbb97bb0da4fc5b0c8e5ab55b9e021, user_id=ad4b191c4db141418794e72abc50248d, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:33.801864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54597f-ac85-4a23-92e8-651889613225 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:33.815280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 607/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:10:33 2022] PUT /identity/v3/projects/e6dbb97bb0da4fc5b0c8e5ab55b9e021/users/ad4b191c4db141418794e72abc50248d/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:10:36.090262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06119839-c765-4818-9481-3d7e07cd248c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:36.090860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06119839-c765-4818-9481-3d7e07cd248c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:36.090860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06119839-c765-4818-9481-3d7e07cd248c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:36.092179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 586/1193] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:10:36.097552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d7522-55f3-4787-a557-24f5bc6595b2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:36.097987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d7522-55f3-4787-a557-24f5bc6595b2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:36.098151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d7522-55f3-4787-a557-24f5bc6595b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:36.114894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8d0d7522-55f3-4787-a557-24f5bc6595b2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:10:36.116089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8d0d7522-55f3-4787-a557-24f5bc6595b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:10:36.181158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 608/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:10:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:10:36.205284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06119839-c765-4818-9481-3d7e07cd248c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:36.313194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06119839-c765-4818-9481-3d7e07cd248c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:36.323242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:36.352758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f29a5-6f9f-4917-af63-0f477a818582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:36.353465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f29a5-6f9f-4917-af63-0f477a818582 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:36.353773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f29a5-6f9f-4917-af63-0f477a818582 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:36.354052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f29a5-6f9f-4917-af63-0f477a818582 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:36.364670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f29a5-6f9f-4917-af63-0f477a818582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-3c8f29a5-6f9f-4917-af63-0f477a818582 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:10:36.372852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:10:36.374088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:10:36.375029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 587/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:10:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:10:36.380288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d7522-55f3-4787-a557-24f5bc6595b2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:36.423681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:36.480277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d7522-55f3-4787-a557-24f5bc6595b2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:36.516237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397762b-7f8c-4ef8-9e7a-881ac73825b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:36.516904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397762b-7f8c-4ef8-9e7a-881ac73825b0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:36.517187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397762b-7f8c-4ef8-9e7a-881ac73825b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:36.517464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397762b-7f8c-4ef8-9e7a-881ac73825b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:36.518252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397762b-7f8c-4ef8-9e7a-881ac73825b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:36.518560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397762b-7f8c-4ef8-9e7a-881ac73825b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:36.519939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397762b-7f8c-4ef8-9e7a-881ac73825b0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:36.531643 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 609/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:10:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:36.537908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f29a5-6f9f-4917-af63-0f477a818582 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:36.649660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f29a5-6f9f-4917-af63-0f477a818582 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:36.683251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29b3df-7f37-49e9-9763-4710c1c3f537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:36.684120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29b3df-7f37-49e9-9763-4710c1c3f537 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:36.684402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29b3df-7f37-49e9-9763-4710c1c3f537 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:36.684675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29b3df-7f37-49e9-9763-4710c1c3f537 None admin] PATH_INFO: `/v3/users/ad4b191c4db141418794e72abc50248d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:36.685896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29b3df-7f37-49e9-9763-4710c1c3f537 None admin] RBAC: Authorizing `identity:get_user(user_id=ad4b191c4db141418794e72abc50248d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:36.687329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29b3df-7f37-49e9-9763-4710c1c3f537 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:36.688955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 588/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 6 14:10:36 2022] GET /identity/v3/users/ad4b191c4db141418794e72abc50248d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:36.695387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397762b-7f8c-4ef8-9e7a-881ac73825b0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:36.796972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397762b-7f8c-4ef8-9e7a-881ac73825b0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:36.799335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:36.833038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1ea32-6798-41c4-9271-87e3ebd6e1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:36.833687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1ea32-6798-41c4-9271-87e3ebd6e1ef None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:36.834020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1ea32-6798-41c4-9271-87e3ebd6e1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:36.834301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1ea32-6798-41c4-9271-87e3ebd6e1ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:36.906284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1ea32-6798-41c4-9271-87e3ebd6e1ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:36.907889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1ea32-6798-41c4-9271-87e3ebd6e1ef None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:37.014568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 610/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:10:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:10:37.020495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29b3df-7f37-49e9-9763-4710c1c3f537 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:37.059084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:37.125145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29b3df-7f37-49e9-9763-4710c1c3f537 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:37.158225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451d4ae-5ded-4fde-9265-2af4a775ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:37.159000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451d4ae-5ded-4fde-9265-2af4a775ef7f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:37.159282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451d4ae-5ded-4fde-9265-2af4a775ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:37.159554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451d4ae-5ded-4fde-9265-2af4a775ef7f None admin] PATH_INFO: `/v3/projects/e6dbb97bb0da4fc5b0c8e5ab55b9e021` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:37.160718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451d4ae-5ded-4fde-9265-2af4a775ef7f None admin] RBAC: Authorizing `identity:get_project(project_id=e6dbb97bb0da4fc5b0c8e5ab55b9e021)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:37.162256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451d4ae-5ded-4fde-9265-2af4a775ef7f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:37.163955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 589/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 6 14:10:37 2022] GET /identity/v3/projects/e6dbb97bb0da4fc5b0c8e5ab55b9e021 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:37.170193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1ea32-6798-41c4-9271-87e3ebd6e1ef None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:10:37.173318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:37.274909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1ea32-6798-41c4-9271-87e3ebd6e1ef None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:10:37.310792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82164c1-6a7d-40df-a164-c5b14e53fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:10:37.311439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82164c1-6a7d-40df-a164-c5b14e53fad8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:10:37.311720 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82164c1-6a7d-40df-a164-c5b14e53fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:10:37.311992 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82164c1-6a7d-40df-a164-c5b14e53fad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:10:37.312820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82164c1-6a7d-40df-a164-c5b14e53fad8 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=e6dbb97bb0da4fc5b0c8e5ab55b9e021, user.id=ad4b191c4db141418794e72abc50248d) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:10:37.313127 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82164c1-6a7d-40df-a164-c5b14e53fad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:10:37.314585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82164c1-6a7d-40df-a164-c5b14e53fad8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:10:37.321587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 611/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:10:37 2022] GET /identity/v3/role_assignments?user.id=ad4b191c4db141418794e72abc50248d&scope.project.id=e6dbb97bb0da4fc5b0c8e5ab55b9e021&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:10:47.153961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=48 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:10:47.318697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=66 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:40.276019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c93cb-4e03-48ca-83e2-b23a0b043112 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:40.276019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c93cb-4e03-48ca-83e2-b23a0b043112 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:40.276019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c93cb-4e03-48ca-83e2-b23a0b043112 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:40.276976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 590/1201] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:11:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:11:40.283918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943db38-e931-409d-9b47-e575e024bb3e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:40.285101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943db38-e931-409d-9b47-e575e024bb3e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:40.286341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943db38-e931-409d-9b47-e575e024bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:40.307896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3943db38-e931-409d-9b47-e575e024bb3e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:11:40.309258 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3943db38-e931-409d-9b47-e575e024bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:11:40.405938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 612/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:11:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:11:40.436788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c93cb-4e03-48ca-83e2-b23a0b043112 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:40.631203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c93cb-4e03-48ca-83e2-b23a0b043112 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:40.684669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347005e9-5086-4ca5-aa8e-7b654ed00cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:40.685510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347005e9-5086-4ca5-aa8e-7b654ed00cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:40.685934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347005e9-5086-4ca5-aa8e-7b654ed00cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:40.686254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347005e9-5086-4ca5-aa8e-7b654ed00cc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:40.702531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347005e9-5086-4ca5-aa8e-7b654ed00cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:40.704531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347005e9-5086-4ca5-aa8e-7b654ed00cc1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-347005e9-5086-4ca5-aa8e-7b654ed00cc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:11:40.731986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:40.733418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:40.735598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:11:40.735598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:11:40.735598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:40.735598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:11:40.735598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 591/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:40.741004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943db38-e931-409d-9b47-e575e024bb3e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:40.894148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943db38-e931-409d-9b47-e575e024bb3e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:40.934650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:40.945873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1860b6-920c-4fc4-b731-56691b80db37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:40.945873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1860b6-920c-4fc4-b731-56691b80db37 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:40.945873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1860b6-920c-4fc4-b731-56691b80db37 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:40.945873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1860b6-920c-4fc4-b731-56691b80db37 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:40.946312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1860b6-920c-4fc4-b731-56691b80db37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:40.946312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1860b6-920c-4fc4-b731-56691b80db37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:40.947731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1860b6-920c-4fc4-b731-56691b80db37 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:40.965130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 613/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:11:40.971608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347005e9-5086-4ca5-aa8e-7b654ed00cc1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:41.055238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:41.140584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347005e9-5086-4ca5-aa8e-7b654ed00cc1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:41.189593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0815d-f70c-4dc0-9292-addbdd514486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:41.190255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0815d-f70c-4dc0-9292-addbdd514486 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:41.190569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0815d-f70c-4dc0-9292-addbdd514486 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:41.190867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0815d-f70c-4dc0-9292-addbdd514486 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:41.191824 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0815d-f70c-4dc0-9292-addbdd514486 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:41.193524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0815d-f70c-4dc0-9292-addbdd514486 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:41.221942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 592/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 6 14:11:40 2022] POST /identity/v3/users => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:11:44.010308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbb4eb-be39-46bd-a98f-2097469ee467 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:44.010832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbb4eb-be39-46bd-a98f-2097469ee467 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:44.010832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbb4eb-be39-46bd-a98f-2097469ee467 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:44.012364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 614/1206] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:11:44.018292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b8f3c-abe3-4095-acae-39758ebecfab None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:44.018788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b8f3c-abe3-4095-acae-39758ebecfab None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:44.018933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b8f3c-abe3-4095-acae-39758ebecfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:44.037514 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-799b8f3c-abe3-4095-acae-39758ebecfab None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:11:44.038832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-799b8f3c-abe3-4095-acae-39758ebecfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:11:44.129744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 593/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:11:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:11:44.155306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbb4eb-be39-46bd-a98f-2097469ee467 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:44.286926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbb4eb-be39-46bd-a98f-2097469ee467 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:44.327071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69313c-a9ee-42a3-8ec5-02a31eb4b896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:44.328050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69313c-a9ee-42a3-8ec5-02a31eb4b896 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:44.328512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69313c-a9ee-42a3-8ec5-02a31eb4b896 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:44.328937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69313c-a9ee-42a3-8ec5-02a31eb4b896 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:44.344130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69313c-a9ee-42a3-8ec5-02a31eb4b896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-cf69313c-a9ee-42a3-8ec5-02a31eb4b896 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:11:44.353036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:44.355263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:44.356146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 615/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:11:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:44.362106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b8f3c-abe3-4095-acae-39758ebecfab None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:44.409284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:44.486945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b8f3c-abe3-4095-acae-39758ebecfab None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:44.526952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c25dc-0f7c-45d8-901c-88875124be55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:44.527626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c25dc-0f7c-45d8-901c-88875124be55 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:44.527919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c25dc-0f7c-45d8-901c-88875124be55 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:44.528249 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c25dc-0f7c-45d8-901c-88875124be55 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:44.529002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c25dc-0f7c-45d8-901c-88875124be55 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:44.529303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c25dc-0f7c-45d8-901c-88875124be55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:44.530689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c25dc-0f7c-45d8-901c-88875124be55 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:44.544946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 594/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:11:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:44.546469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69313c-a9ee-42a3-8ec5-02a31eb4b896 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:44.610248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:44.675872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69313c-a9ee-42a3-8ec5-02a31eb4b896 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:44.723297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d8f0e-f27c-4aeb-bea9-1918efb08f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:44.724362 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d8f0e-f27c-4aeb-bea9-1918efb08f83 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:44.724623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d8f0e-f27c-4aeb-bea9-1918efb08f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:44.724888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d8f0e-f27c-4aeb-bea9-1918efb08f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:44.738223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d8f0e-f27c-4aeb-bea9-1918efb08f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:44.740024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d8f0e-f27c-4aeb-bea9-1918efb08f83 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-994d8f0e-f27c-4aeb-bea9-1918efb08f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:11:44.759685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:44.760693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:44.761647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:11:44.761647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:11:44.761647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:44.761647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:44.761647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 616/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:11:44.767756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c25dc-0f7c-45d8-901c-88875124be55 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:44.893950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:44.898711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c25dc-0f7c-45d8-901c-88875124be55 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:44.954455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e55fe-e897-4a27-bd59-b9f6b5d68a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:44.955757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e55fe-e897-4a27-bd59-b9f6b5d68a67 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:44.955757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e55fe-e897-4a27-bd59-b9f6b5d68a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:44.955757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e55fe-e897-4a27-bd59-b9f6b5d68a67 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:44.956403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e55fe-e897-4a27-bd59-b9f6b5d68a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:44.956712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e55fe-e897-4a27-bd59-b9f6b5d68a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:44.958666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e55fe-e897-4a27-bd59-b9f6b5d68a67 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:44.977008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 595/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:44.982587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d8f0e-f27c-4aeb-bea9-1918efb08f83 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:45.095151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d8f0e-f27c-4aeb-bea9-1918efb08f83 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:45.096999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:45.137097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ddd6d-2693-476d-ac4b-016fff46c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:45.138504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ddd6d-2693-476d-ac4b-016fff46c253 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:45.138504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ddd6d-2693-476d-ac4b-016fff46c253 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:45.138504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ddd6d-2693-476d-ac4b-016fff46c253 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:45.147616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ddd6d-2693-476d-ac4b-016fff46c253 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:45.148126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ddd6d-2693-476d-ac4b-016fff46c253 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c63ddd6d-2693-476d-ac4b-016fff46c253 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:11:45.164400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:45.165987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:45.168114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 617/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:11:45.171479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e55fe-e897-4a27-bd59-b9f6b5d68a67 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:45.287858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e55fe-e897-4a27-bd59-b9f6b5d68a67 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:45.327886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24762a28-6158-4b44-b58f-242f48e2db8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:45.328575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24762a28-6158-4b44-b58f-242f48e2db8d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:45.328901 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24762a28-6158-4b44-b58f-242f48e2db8d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:45.329244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24762a28-6158-4b44-b58f-242f48e2db8d None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:45.330526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24762a28-6158-4b44-b58f-242f48e2db8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:45.330878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24762a28-6158-4b44-b58f-242f48e2db8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:45.333259 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24762a28-6158-4b44-b58f-242f48e2db8d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:45.361142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 596/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 6 14:11:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:45.362418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ddd6d-2693-476d-ac4b-016fff46c253 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:45.485957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ddd6d-2693-476d-ac4b-016fff46c253 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:45.523287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805b96a-4f42-4e81-a791-1e27633eb88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:45.523939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805b96a-4f42-4e81-a791-1e27633eb88a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:45.524226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805b96a-4f42-4e81-a791-1e27633eb88a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:45.524505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805b96a-4f42-4e81-a791-1e27633eb88a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:45.558314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:45.603564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805b96a-4f42-4e81-a791-1e27633eb88a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:45.605079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805b96a-4f42-4e81-a791-1e27633eb88a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:45.748018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 618/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:11:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:11:45.755296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24762a28-6158-4b44-b58f-242f48e2db8d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:45.780397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:45.886967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24762a28-6158-4b44-b58f-242f48e2db8d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:45.928165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b24d4-c3a5-4b42-92e0-c0e0d7b27516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:45.929253 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b24d4-c3a5-4b42-92e0-c0e0d7b27516 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:45.929729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b24d4-c3a5-4b42-92e0-c0e0d7b27516 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:45.930152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b24d4-c3a5-4b42-92e0-c0e0d7b27516 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:45.942944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b24d4-c3a5-4b42-92e0-c0e0d7b27516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:45.945232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b24d4-c3a5-4b42-92e0-c0e0d7b27516 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-546b24d4-c3a5-4b42-92e0-c0e0d7b27516 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:11:45.970171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:45.971628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:45.973393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:11:45.973393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:11:45.973393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:45.973393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:11:45.975228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 597/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:45.981629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805b96a-4f42-4e81-a791-1e27633eb88a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:46.060985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:46.113057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805b96a-4f42-4e81-a791-1e27633eb88a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:46.153763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a491728-6c0e-4501-baaf-f64abe381f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:46.154460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a491728-6c0e-4501-baaf-f64abe381f4e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:46.154742 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a491728-6c0e-4501-baaf-f64abe381f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:46.155018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a491728-6c0e-4501-baaf-f64abe381f4e None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:46.155768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a491728-6c0e-4501-baaf-f64abe381f4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:46.156078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a491728-6c0e-4501-baaf-f64abe381f4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:46.157516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a491728-6c0e-4501-baaf-f64abe381f4e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:46.171001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 619/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:46.178575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b24d4-c3a5-4b42-92e0-c0e0d7b27516 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:46.295828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:46.330950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b24d4-c3a5-4b42-92e0-c0e0d7b27516 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:46.372921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfdbd-92a0-4a6a-ad28-3b121ae1c57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:46.374289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfdbd-92a0-4a6a-ad28-3b121ae1c57e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:46.374727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfdbd-92a0-4a6a-ad28-3b121ae1c57e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:46.375217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfdbd-92a0-4a6a-ad28-3b121ae1c57e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:46.385630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfdbd-92a0-4a6a-ad28-3b121ae1c57e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:46.387907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfdbd-92a0-4a6a-ad28-3b121ae1c57e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f31bfdbd-92a0-4a6a-ad28-3b121ae1c57e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:11:46.406555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:46.407622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:46.408560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:11:46.408560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:11:46.408560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:11:46.408560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:11:46.408560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:11:46.408560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:11:46.408560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:11:46.408560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:11:46.414553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a491728-6c0e-4501-baaf-f64abe381f4e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:46.417333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 598/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:11:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:46.561408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a491728-6c0e-4501-baaf-f64abe381f4e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:46.596174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:46.610981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fdb03-beb8-4698-9962-35a74e52f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:46.611911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fdb03-beb8-4698-9962-35a74e52f2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:46.612192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fdb03-beb8-4698-9962-35a74e52f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:46.612496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fdb03-beb8-4698-9962-35a74e52f2e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:46.613974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fdb03-beb8-4698-9962-35a74e52f2e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:46.614267 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fdb03-beb8-4698-9962-35a74e52f2e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:46.616322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fdb03-beb8-4698-9962-35a74e52f2e0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:46.632255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 620/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:11:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:11:46.638051 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfdbd-92a0-4a6a-ad28-3b121ae1c57e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:46.776913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfdbd-92a0-4a6a-ad28-3b121ae1c57e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:46.813245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:46.820975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711eaa06-ac65-4b0d-8a96-c777c0766c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:46.821694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eaa06-ac65-4b0d-8a96-c777c0766c64 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:46.822086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eaa06-ac65-4b0d-8a96-c777c0766c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:46.822368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eaa06-ac65-4b0d-8a96-c777c0766c64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:46.823363 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711eaa06-ac65-4b0d-8a96-c777c0766c64 None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=51b91456a0634e0fa22fb389f0721a65) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:46.823683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711eaa06-ac65-4b0d-8a96-c777c0766c64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:46.825418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711eaa06-ac65-4b0d-8a96-c777c0766c64 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:46.833100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 599/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:11:46 2022] GET /identity/v3/role_assignments?user.id=51b91456a0634e0fa22fb389f0721a65&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:49.460056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106ce62-5975-47b7-b3fe-9daa85d416d7 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:49.460590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106ce62-5975-47b7-b3fe-9daa85d416d7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:49.460590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106ce62-5975-47b7-b3fe-9daa85d416d7 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:49.462034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 621/1220] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:11:49.467333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192bb0d-a602-439d-802e-7d80711e1bb4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:49.467751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192bb0d-a602-439d-802e-7d80711e1bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:49.467888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192bb0d-a602-439d-802e-7d80711e1bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:49.485060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c192bb0d-a602-439d-802e-7d80711e1bb4 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:11:49.486281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c192bb0d-a602-439d-802e-7d80711e1bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:11:49.583474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 600/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:11:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:11:49.612741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106ce62-5975-47b7-b3fe-9daa85d416d7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:49.746707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106ce62-5975-47b7-b3fe-9daa85d416d7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:49.787740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68d5dc-419f-42ef-ae6f-d468576f00e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:49.788672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68d5dc-419f-42ef-ae6f-d468576f00e3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:49.788975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68d5dc-419f-42ef-ae6f-d468576f00e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:49.789254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68d5dc-419f-42ef-ae6f-d468576f00e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:49.803088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68d5dc-419f-42ef-ae6f-d468576f00e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-9e68d5dc-419f-42ef-ae6f-d468576f00e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:11:49.811187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:49.812457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:49.813466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 622/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:11:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:49.820020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192bb0d-a602-439d-802e-7d80711e1bb4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:49.961075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192bb0d-a602-439d-802e-7d80711e1bb4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:49.991564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:50.006693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36dec6f-1cb7-4a0e-b3d1-6b3aa17ba797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:50.007603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36dec6f-1cb7-4a0e-b3d1-6b3aa17ba797 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:50.007933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36dec6f-1cb7-4a0e-b3d1-6b3aa17ba797 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:50.011759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36dec6f-1cb7-4a0e-b3d1-6b3aa17ba797 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:50.011759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36dec6f-1cb7-4a0e-b3d1-6b3aa17ba797 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:50.011759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36dec6f-1cb7-4a0e-b3d1-6b3aa17ba797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:50.011759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36dec6f-1cb7-4a0e-b3d1-6b3aa17ba797 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:50.025036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 601/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:11:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:50.031992 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68d5dc-419f-42ef-ae6f-d468576f00e3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:50.101194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:50.186864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68d5dc-419f-42ef-ae6f-d468576f00e3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:50.242228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a889fe-5d73-4ebe-b683-6654bed1b99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:50.242228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a889fe-5d73-4ebe-b683-6654bed1b99d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:50.242228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a889fe-5d73-4ebe-b683-6654bed1b99d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:50.242655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a889fe-5d73-4ebe-b683-6654bed1b99d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:50.257141 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a889fe-5d73-4ebe-b683-6654bed1b99d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:50.258602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a889fe-5d73-4ebe-b683-6654bed1b99d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-66a889fe-5d73-4ebe-b683-6654bed1b99d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:11:50.281391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:50.282665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:50.283612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:11:50.283612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:11:50.283612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:50.283612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:50.283612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 623/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:11:50.286693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36dec6f-1cb7-4a0e-b3d1-6b3aa17ba797 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:50.447830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36dec6f-1cb7-4a0e-b3d1-6b3aa17ba797 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:50.494843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d9b36-f0d2-4c17-8506-3a662fb156c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:50.495881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d9b36-f0d2-4c17-8506-3a662fb156c3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:50.496440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d9b36-f0d2-4c17-8506-3a662fb156c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:50.496874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d9b36-f0d2-4c17-8506-3a662fb156c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:50.498001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d9b36-f0d2-4c17-8506-3a662fb156c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:50.498373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d9b36-f0d2-4c17-8506-3a662fb156c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:50.499851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d9b36-f0d2-4c17-8506-3a662fb156c3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:50.517885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 602/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:11:50.524846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a889fe-5d73-4ebe-b683-6654bed1b99d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:50.641148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:50.647438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a889fe-5d73-4ebe-b683-6654bed1b99d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:50.692150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcf634-0900-4e90-b226-99a50fd8c894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:50.698122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcf634-0900-4e90-b226-99a50fd8c894 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:50.698461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcf634-0900-4e90-b226-99a50fd8c894 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:50.698875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcf634-0900-4e90-b226-99a50fd8c894 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:50.709106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcf634-0900-4e90-b226-99a50fd8c894 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:50.710835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcf634-0900-4e90-b226-99a50fd8c894 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-cadcf634-0900-4e90-b226-99a50fd8c894 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:11:50.728923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:50.730412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:50.731696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 624/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:50.736995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d9b36-f0d2-4c17-8506-3a662fb156c3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:50.805107 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:50.876865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d9b36-f0d2-4c17-8506-3a662fb156c3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:50.917997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4189c-02c8-4b1a-9766-8d00dece8b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:50.918684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4189c-02c8-4b1a-9766-8d00dece8b97 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:50.919040 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4189c-02c8-4b1a-9766-8d00dece8b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:50.919322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4189c-02c8-4b1a-9766-8d00dece8b97 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:50.920172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4189c-02c8-4b1a-9766-8d00dece8b97 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:50.920479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4189c-02c8-4b1a-9766-8d00dece8b97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:50.922779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4189c-02c8-4b1a-9766-8d00dece8b97 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:50.945878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 603/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 6 14:11:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:50.951516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcf634-0900-4e90-b226-99a50fd8c894 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:51.077394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcf634-0900-4e90-b226-99a50fd8c894 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:51.116302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3323519-88aa-4bb2-9d27-bf1fbb3f0f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:51.117101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3323519-88aa-4bb2-9d27-bf1fbb3f0f54 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:51.117379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3323519-88aa-4bb2-9d27-bf1fbb3f0f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:51.117651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3323519-88aa-4bb2-9d27-bf1fbb3f0f54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:51.131358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3323519-88aa-4bb2-9d27-bf1fbb3f0f54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:51.132779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3323519-88aa-4bb2-9d27-bf1fbb3f0f54 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b3323519-88aa-4bb2-9d27-bf1fbb3f0f54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:11:51.154020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:51.155207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:51.156249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:11:51.156249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:11:51.156249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:51.156249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:51.156249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 625/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:51.161698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4189c-02c8-4b1a-9766-8d00dece8b97 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:51.272257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4189c-02c8-4b1a-9766-8d00dece8b97 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:51.274811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:51.311803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935a764-e4c5-4f74-8c87-3ca54cf16466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:51.312463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935a764-e4c5-4f74-8c87-3ca54cf16466 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:51.312747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935a764-e4c5-4f74-8c87-3ca54cf16466 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:51.313020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935a764-e4c5-4f74-8c87-3ca54cf16466 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:51.313782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935a764-e4c5-4f74-8c87-3ca54cf16466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:51.314369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935a764-e4c5-4f74-8c87-3ca54cf16466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:51.315762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935a764-e4c5-4f74-8c87-3ca54cf16466 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:51.328354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 604/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:11:51.335653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3323519-88aa-4bb2-9d27-bf1fbb3f0f54 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:51.345237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:51.448841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3323519-88aa-4bb2-9d27-bf1fbb3f0f54 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:51.483120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33e2eb-43e0-4597-8c97-f47860f8800e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:51.483920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33e2eb-43e0-4597-8c97-f47860f8800e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:51.484207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33e2eb-43e0-4597-8c97-f47860f8800e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:51.484486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33e2eb-43e0-4597-8c97-f47860f8800e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:51.492057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33e2eb-43e0-4597-8c97-f47860f8800e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:51.493507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33e2eb-43e0-4597-8c97-f47860f8800e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2f33e2eb-43e0-4597-8c97-f47860f8800e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:11:51.508951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:51.510165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:51.511583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:11:51.511583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:11:51.511583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:11:51.511583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:11:51.511583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:11:51.511583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:11:51.511583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:11:51.511583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:51.511583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 626/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:11:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:51.516757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935a764-e4c5-4f74-8c87-3ca54cf16466 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:51.630708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935a764-e4c5-4f74-8c87-3ca54cf16466 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:51.666279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f6687-a61d-496b-abbd-78db0859af5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:51.666945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f6687-a61d-496b-abbd-78db0859af5d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:51.667226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f6687-a61d-496b-abbd-78db0859af5d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:51.667534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f6687-a61d-496b-abbd-78db0859af5d None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:51.668284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f6687-a61d-496b-abbd-78db0859af5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:51.668595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f6687-a61d-496b-abbd-78db0859af5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:51.670045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f6687-a61d-496b-abbd-78db0859af5d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:51.682996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 605/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:11:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:51.689305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33e2eb-43e0-4597-8c97-f47860f8800e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:51.768551 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:51.809289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33e2eb-43e0-4597-8c97-f47860f8800e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:51.846864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7dd0d-f610-456d-8975-48ba58141a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:51.849290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7dd0d-f610-456d-8975-48ba58141a6b None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:51.849567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7dd0d-f610-456d-8975-48ba58141a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:51.849903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7dd0d-f610-456d-8975-48ba58141a6b None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/51b91456a0634e0fa22fb389f0721a65/roles/4107928942534d3984671dc5232d061d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:51.865412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7dd0d-f610-456d-8975-48ba58141a6b None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=51b91456a0634e0fa22fb389f0721a65, role_id=4107928942534d3984671dc5232d061d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:51.866912 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7dd0d-f610-456d-8975-48ba58141a6b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:51.880841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 627/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:11:51 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/51b91456a0634e0fa22fb389f0721a65/roles/4107928942534d3984671dc5232d061d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:11:54.364539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2423129-53d4-4a5f-a4fc-d8070de6240a None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:54.365124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2423129-53d4-4a5f-a4fc-d8070de6240a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:54.365234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2423129-53d4-4a5f-a4fc-d8070de6240a None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:54.366725 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 606/1233] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:11:54.372462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb8c6-a23b-4f9e-97c8-9a1d4dc53e12 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:54.372916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb8c6-a23b-4f9e-97c8-9a1d4dc53e12 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:54.373010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb8c6-a23b-4f9e-97c8-9a1d4dc53e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:54.390001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d87cb8c6-a23b-4f9e-97c8-9a1d4dc53e12 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:11:54.391210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d87cb8c6-a23b-4f9e-97c8-9a1d4dc53e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:11:54.471620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 628/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:11:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:11:54.498324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2423129-53d4-4a5f-a4fc-d8070de6240a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:54.529321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:54.632136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2423129-53d4-4a5f-a4fc-d8070de6240a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:54.675261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13266aa-8b20-46d8-9fe3-e89e08993972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:54.676359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13266aa-8b20-46d8-9fe3-e89e08993972 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:54.676639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13266aa-8b20-46d8-9fe3-e89e08993972 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:54.676904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13266aa-8b20-46d8-9fe3-e89e08993972 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:54.688956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13266aa-8b20-46d8-9fe3-e89e08993972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c13266aa-8b20-46d8-9fe3-e89e08993972 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:11:54.697221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:54.698485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:11:54.699409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 607/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 6 14:11:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:54.704832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb8c6-a23b-4f9e-97c8-9a1d4dc53e12 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:54.783081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:54.846630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb8c6-a23b-4f9e-97c8-9a1d4dc53e12 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:54.887246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f0bd8-fa9f-4b3e-884f-461ea2fdd713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:54.887855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f0bd8-fa9f-4b3e-884f-461ea2fdd713 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:54.888158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f0bd8-fa9f-4b3e-884f-461ea2fdd713 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:54.888443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f0bd8-fa9f-4b3e-884f-461ea2fdd713 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:54.889191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f0bd8-fa9f-4b3e-884f-461ea2fdd713 None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:54.889500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f0bd8-fa9f-4b3e-884f-461ea2fdd713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:54.890950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f0bd8-fa9f-4b3e-884f-461ea2fdd713 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:54.902489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 629/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 6 14:11:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:54.908463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13266aa-8b20-46d8-9fe3-e89e08993972 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:55.013447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:55.032913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13266aa-8b20-46d8-9fe3-e89e08993972 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:55.080186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daff6ec-a7ac-484b-a3fc-5dd6b78a75fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:55.081125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daff6ec-a7ac-484b-a3fc-5dd6b78a75fc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:55.081475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daff6ec-a7ac-484b-a3fc-5dd6b78a75fc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:55.081793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daff6ec-a7ac-484b-a3fc-5dd6b78a75fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:55.093668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daff6ec-a7ac-484b-a3fc-5dd6b78a75fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:55.095134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daff6ec-a7ac-484b-a3fc-5dd6b78a75fc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7daff6ec-a7ac-484b-a3fc-5dd6b78a75fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:11:55.115142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:55.116500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:55.117604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:11:55.117604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:11:55.117604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:55.117604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:11:55.121199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 608/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:55.122296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f0bd8-fa9f-4b3e-884f-461ea2fdd713 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:55.243582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f0bd8-fa9f-4b3e-884f-461ea2fdd713 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:55.255302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:55.283239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060024c2-58fa-476d-aa2c-2564c7dd5452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:55.283905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060024c2-58fa-476d-aa2c-2564c7dd5452 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:55.284182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060024c2-58fa-476d-aa2c-2564c7dd5452 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:55.284514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060024c2-58fa-476d-aa2c-2564c7dd5452 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:55.285485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060024c2-58fa-476d-aa2c-2564c7dd5452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:55.285816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060024c2-58fa-476d-aa2c-2564c7dd5452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:55.287444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060024c2-58fa-476d-aa2c-2564c7dd5452 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:55.301119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 630/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:11:55.307131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daff6ec-a7ac-484b-a3fc-5dd6b78a75fc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:55.423854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daff6ec-a7ac-484b-a3fc-5dd6b78a75fc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:55.462044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de6e7c-3329-42c1-b44f-adc509cb0bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:55.462919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de6e7c-3329-42c1-b44f-adc509cb0bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:55.463285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de6e7c-3329-42c1-b44f-adc509cb0bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:55.463601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de6e7c-3329-42c1-b44f-adc509cb0bc3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:55.467100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:55.474094 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de6e7c-3329-42c1-b44f-adc509cb0bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:55.476062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de6e7c-3329-42c1-b44f-adc509cb0bc3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f3de6e7c-3329-42c1-b44f-adc509cb0bc3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:11:55.491434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:55.492598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:11:55.493640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 609/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:55.498544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060024c2-58fa-476d-aa2c-2564c7dd5452 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:55.610681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060024c2-58fa-476d-aa2c-2564c7dd5452 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:55.646207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a830f-27c7-4234-9ccd-26c14d3e148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:55.646905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a830f-27c7-4234-9ccd-26c14d3e148b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:55.647241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a830f-27c7-4234-9ccd-26c14d3e148b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:55.647581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a830f-27c7-4234-9ccd-26c14d3e148b None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:55.648451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a830f-27c7-4234-9ccd-26c14d3e148b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:55.648870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a830f-27c7-4234-9ccd-26c14d3e148b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:55.650341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a830f-27c7-4234-9ccd-26c14d3e148b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:55.668948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 631/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 6 14:11:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:55.674826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de6e7c-3329-42c1-b44f-adc509cb0bc3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:55.794891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de6e7c-3329-42c1-b44f-adc509cb0bc3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:55.837975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba47128-cdaa-4624-8641-9f9bd6b2d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:55.838620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba47128-cdaa-4624-8641-9f9bd6b2d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:55.838905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba47128-cdaa-4624-8641-9f9bd6b2d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:55.839242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba47128-cdaa-4624-8641-9f9bd6b2d0d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:55.908698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:55.917055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba47128-cdaa-4624-8641-9f9bd6b2d0d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:55.918601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba47128-cdaa-4624-8641-9f9bd6b2d0d5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:56.030967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 610/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:11:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:11:56.037427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a830f-27c7-4234-9ccd-26c14d3e148b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:56.068067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:56.159567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a830f-27c7-4234-9ccd-26c14d3e148b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:56.198330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f32493-808f-4759-bca5-9b16e02a5637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:56.199131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f32493-808f-4759-bca5-9b16e02a5637 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:56.199417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f32493-808f-4759-bca5-9b16e02a5637 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:56.199691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f32493-808f-4759-bca5-9b16e02a5637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:56.210533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f32493-808f-4759-bca5-9b16e02a5637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:56.212144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f32493-808f-4759-bca5-9b16e02a5637 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-88f32493-808f-4759-bca5-9b16e02a5637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:11:56.231704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:56.232848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:56.234130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:11:56.234130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:11:56.234130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:11:56.234130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:56.234130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 632/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:56.239485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba47128-cdaa-4624-8641-9f9bd6b2d0d5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:56.349587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:56.356128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba47128-cdaa-4624-8641-9f9bd6b2d0d5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:56.391343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bea34-3052-4c7b-bd6a-6cd6f86c4d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:56.392002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bea34-3052-4c7b-bd6a-6cd6f86c4d10 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:56.392279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bea34-3052-4c7b-bd6a-6cd6f86c4d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:56.392548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bea34-3052-4c7b-bd6a-6cd6f86c4d10 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:56.393285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bea34-3052-4c7b-bd6a-6cd6f86c4d10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:56.393588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bea34-3052-4c7b-bd6a-6cd6f86c4d10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:56.395783 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bea34-3052-4c7b-bd6a-6cd6f86c4d10 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:56.407471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 611/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:56.413788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f32493-808f-4759-bca5-9b16e02a5637 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:56.496987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:56.526316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f32493-808f-4759-bca5-9b16e02a5637 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:56.567843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da47e7-2248-44cf-9d2b-633a89988cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:56.568646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da47e7-2248-44cf-9d2b-633a89988cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:56.568935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da47e7-2248-44cf-9d2b-633a89988cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:56.569282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da47e7-2248-44cf-9d2b-633a89988cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:56.577613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da47e7-2248-44cf-9d2b-633a89988cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:56.579434 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da47e7-2248-44cf-9d2b-633a89988cf8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-34da47e7-2248-44cf-9d2b-633a89988cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:11:56.597825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:11:56.599257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:11:56.600564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:11:56.600564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:11:56.600564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:11:56.600564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:11:56.600564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:11:56.600564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:11:56.600564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:11:56.600564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:11:56.600564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 633/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:11:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:11:56.607327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bea34-3052-4c7b-bd6a-6cd6f86c4d10 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:56.714761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bea34-3052-4c7b-bd6a-6cd6f86c4d10 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:56.758981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45dbdd-67ea-41a4-8d20-fca8ff74d7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:56.760076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45dbdd-67ea-41a4-8d20-fca8ff74d7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:56.760414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45dbdd-67ea-41a4-8d20-fca8ff74d7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:56.760808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45dbdd-67ea-41a4-8d20-fca8ff74d7c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:56.762144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45dbdd-67ea-41a4-8d20-fca8ff74d7c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:56.762703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45dbdd-67ea-41a4-8d20-fca8ff74d7c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:56.765232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45dbdd-67ea-41a4-8d20-fca8ff74d7c6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:56.784007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 612/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:11:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:56.790464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da47e7-2248-44cf-9d2b-633a89988cf8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:56.909842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da47e7-2248-44cf-9d2b-633a89988cf8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:56.948937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324968b-a6a0-4136-abd8-c47d0077afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:56.949599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324968b-a6a0-4136-abd8-c47d0077afb8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:56.949916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324968b-a6a0-4136-abd8-c47d0077afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:56.950198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324968b-a6a0-4136-abd8-c47d0077afb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:56.951035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324968b-a6a0-4136-abd8-c47d0077afb8 None admin] RBAC: Adding query filter params (role.id=4107928942534d3984671dc5232d061d, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=51b91456a0634e0fa22fb389f0721a65) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:11:56.951351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324968b-a6a0-4136-abd8-c47d0077afb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:11:56.952828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324968b-a6a0-4136-abd8-c47d0077afb8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:11:56.960358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 634/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:11:56 2022] GET /identity/v3/role_assignments?user.id=51b91456a0634e0fa22fb389f0721a65&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=4107928942534d3984671dc5232d061d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:11:59.567237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30633d-e75f-4d4b-a99e-cad68fe68d01 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:59.567920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30633d-e75f-4d4b-a99e-cad68fe68d01 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:59.568488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30633d-e75f-4d4b-a99e-cad68fe68d01 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:59.571337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 613/1247] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:11:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:11:59.578424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f8947-59db-42c8-b8e7-c34ac102c896 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:59.578958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f8947-59db-42c8-b8e7-c34ac102c896 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:59.579150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f8947-59db-42c8-b8e7-c34ac102c896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:59.587337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:59.602556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d50f8947-59db-42c8-b8e7-c34ac102c896 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:11:59.603850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d50f8947-59db-42c8-b8e7-c34ac102c896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:11:59.708309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 635/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:11:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 14:11:59.742717 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30633d-e75f-4d4b-a99e-cad68fe68d01 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:11:59.773898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:11:59.919593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30633d-e75f-4d4b-a99e-cad68fe68d01 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:11:59.975053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a1072-f928-4c6a-bbc7-3f466b3448a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:11:59.975916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a1072-f928-4c6a-bbc7-3f466b3448a1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:11:59.976201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a1072-f928-4c6a-bbc7-3f466b3448a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:11:59.976529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a1072-f928-4c6a-bbc7-3f466b3448a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:11:59.991898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a1072-f928-4c6a-bbc7-3f466b3448a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d52a1072-f928-4c6a-bbc7-3f466b3448a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:12:00.000066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:00.002174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:00.004412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 614/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:12:00.009626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f8947-59db-42c8-b8e7-c34ac102c896 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:00.190004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f8947-59db-42c8-b8e7-c34ac102c896 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:00.232402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:00.243128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481335b-2fae-47d0-a5c4-0c5f3723aae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:00.244361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481335b-2fae-47d0-a5c4-0c5f3723aae1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:00.245115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481335b-2fae-47d0-a5c4-0c5f3723aae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:00.245740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481335b-2fae-47d0-a5c4-0c5f3723aae1 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:00.246659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481335b-2fae-47d0-a5c4-0c5f3723aae1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:00.247084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481335b-2fae-47d0-a5c4-0c5f3723aae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:00.249411 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481335b-2fae-47d0-a5c4-0c5f3723aae1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:00.264929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 636/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:12:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:00.274589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a1072-f928-4c6a-bbc7-3f466b3448a1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:00.395555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:00.434360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a1072-f928-4c6a-bbc7-3f466b3448a1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:00.495422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45747018-eb4f-4aa7-b34a-0d3ada28bd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:00.496279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45747018-eb4f-4aa7-b34a-0d3ada28bd17 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:00.496606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45747018-eb4f-4aa7-b34a-0d3ada28bd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:00.496931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45747018-eb4f-4aa7-b34a-0d3ada28bd17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:00.517137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45747018-eb4f-4aa7-b34a-0d3ada28bd17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:00.519716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45747018-eb4f-4aa7-b34a-0d3ada28bd17 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-45747018-eb4f-4aa7-b34a-0d3ada28bd17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:12:00.541639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:00.543942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:00.546030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:12:00.546030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:12:00.546030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:00.546030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:00.546030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 615/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:12:00.553295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481335b-2fae-47d0-a5c4-0c5f3723aae1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:00.711720 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481335b-2fae-47d0-a5c4-0c5f3723aae1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:00.771269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00174-4dc7-42be-9230-1cdd52be5dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:00.771942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00174-4dc7-42be-9230-1cdd52be5dae None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:00.772601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00174-4dc7-42be-9230-1cdd52be5dae None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:00.772773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00174-4dc7-42be-9230-1cdd52be5dae None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:00.773782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c00174-4dc7-42be-9230-1cdd52be5dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:00.774209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c00174-4dc7-42be-9230-1cdd52be5dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:00.776018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c00174-4dc7-42be-9230-1cdd52be5dae None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:00.793454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 637/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:00.803461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45747018-eb4f-4aa7-b34a-0d3ada28bd17 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:00.946016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45747018-eb4f-4aa7-b34a-0d3ada28bd17 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:00.989646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:01.002122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c372aee-5143-4319-9dea-f5368c789782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:01.003003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c372aee-5143-4319-9dea-f5368c789782 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:01.004368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c372aee-5143-4319-9dea-f5368c789782 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:01.004368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c372aee-5143-4319-9dea-f5368c789782 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:01.020827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c372aee-5143-4319-9dea-f5368c789782 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:01.024688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c372aee-5143-4319-9dea-f5368c789782 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9c372aee-5143-4319-9dea-f5368c789782 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:12:01.047735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:01.049808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:01.051856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 616/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:12:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:01.056766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00174-4dc7-42be-9230-1cdd52be5dae None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:01.157871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:01.239784 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00174-4dc7-42be-9230-1cdd52be5dae None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:01.289196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a11171-9f0f-4ddb-8c6e-462632c3a4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:01.289852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a11171-9f0f-4ddb-8c6e-462632c3a4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:01.290154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a11171-9f0f-4ddb-8c6e-462632c3a4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:01.290414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a11171-9f0f-4ddb-8c6e-462632c3a4c6 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:01.291394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a11171-9f0f-4ddb-8c6e-462632c3a4c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:01.291711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a11171-9f0f-4ddb-8c6e-462632c3a4c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:01.293222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a11171-9f0f-4ddb-8c6e-462632c3a4c6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:01.317603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 638/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 6 14:12:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:01.324077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c372aee-5143-4319-9dea-f5368c789782 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:01.476806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c372aee-5143-4319-9dea-f5368c789782 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:01.524321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55569e-6661-47b6-9241-4dc4b8ce21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:01.525484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55569e-6661-47b6-9241-4dc4b8ce21fa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:01.526035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55569e-6661-47b6-9241-4dc4b8ce21fa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:01.526534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55569e-6661-47b6-9241-4dc4b8ce21fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:01.621691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:01.641273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55569e-6661-47b6-9241-4dc4b8ce21fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:01.643011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55569e-6661-47b6-9241-4dc4b8ce21fa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:01.822956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 617/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:12:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 14:12:01.834005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a11171-9f0f-4ddb-8c6e-462632c3a4c6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:01.985919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a11171-9f0f-4ddb-8c6e-462632c3a4c6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:02.006700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:02.047433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d766f-3f92-43fb-8e94-fb19cf8404e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:02.048886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d766f-3f92-43fb-8e94-fb19cf8404e5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:02.049311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d766f-3f92-43fb-8e94-fb19cf8404e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:02.049694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d766f-3f92-43fb-8e94-fb19cf8404e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:02.070475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d766f-3f92-43fb-8e94-fb19cf8404e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:02.073365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d766f-3f92-43fb-8e94-fb19cf8404e5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d35d766f-3f92-43fb-8e94-fb19cf8404e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:12:02.106576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:02.108444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:02.110283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:12:02.110283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:12:02.110283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:02.110283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:02.110283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 639/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:02.121497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55569e-6661-47b6-9241-4dc4b8ce21fa None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:02.290821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:02.308343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55569e-6661-47b6-9241-4dc4b8ce21fa None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:02.356210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5557d8-d058-40ed-b85f-cc3804f54699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:02.356804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5557d8-d058-40ed-b85f-cc3804f54699 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:02.357127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5557d8-d058-40ed-b85f-cc3804f54699 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:02.357555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5557d8-d058-40ed-b85f-cc3804f54699 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:02.358365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5557d8-d058-40ed-b85f-cc3804f54699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:02.358703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5557d8-d058-40ed-b85f-cc3804f54699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:02.360293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5557d8-d058-40ed-b85f-cc3804f54699 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:02.376288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 618/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:02.383210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d766f-3f92-43fb-8e94-fb19cf8404e5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:02.537295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d766f-3f92-43fb-8e94-fb19cf8404e5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:02.585680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35fcb1-239d-4cb2-9ebe-c882234a3468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:02.586666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35fcb1-239d-4cb2-9ebe-c882234a3468 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:02.587098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35fcb1-239d-4cb2-9ebe-c882234a3468 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:02.587420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35fcb1-239d-4cb2-9ebe-c882234a3468 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:02.596241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35fcb1-239d-4cb2-9ebe-c882234a3468 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:02.599841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35fcb1-239d-4cb2-9ebe-c882234a3468 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-4f35fcb1-239d-4cb2-9ebe-c882234a3468 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:12:02.628431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:02.629608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:02.630737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:12:02.630737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:12:02.630737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:12:02.630737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:12:02.630737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:12:02.630737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:12:02.630737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:12:02.630737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:02.631685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 640/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:12:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:02.641333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5557d8-d058-40ed-b85f-cc3804f54699 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:02.798395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5557d8-d058-40ed-b85f-cc3804f54699 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:02.845927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766d71b-6a9e-4b6f-8bae-a1797f4d6845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:02.846707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766d71b-6a9e-4b6f-8bae-a1797f4d6845 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:02.846867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766d71b-6a9e-4b6f-8bae-a1797f4d6845 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:02.847140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766d71b-6a9e-4b6f-8bae-a1797f4d6845 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:02.849236 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766d71b-6a9e-4b6f-8bae-a1797f4d6845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:02.849578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766d71b-6a9e-4b6f-8bae-a1797f4d6845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:02.851011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766d71b-6a9e-4b6f-8bae-a1797f4d6845 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:02.866421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 619/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:12:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:02.872954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35fcb1-239d-4cb2-9ebe-c882234a3468 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:02.903454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:02.992100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35fcb1-239d-4cb2-9ebe-c882234a3468 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:03.035476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed6562-c1f7-4ba4-80cf-15c49cb86d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:03.037041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed6562-c1f7-4ba4-80cf-15c49cb86d88 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:03.037362 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed6562-c1f7-4ba4-80cf-15c49cb86d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:03.037645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed6562-c1f7-4ba4-80cf-15c49cb86d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:03.038710 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed6562-c1f7-4ba4-80cf-15c49cb86d88 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=51b91456a0634e0fa22fb389f0721a65) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:03.039029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed6562-c1f7-4ba4-80cf-15c49cb86d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:03.040948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed6562-c1f7-4ba4-80cf-15c49cb86d88 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:03.048378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 641/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:12:02 2022] GET /identity/v3/role_assignments?user.id=51b91456a0634e0fa22fb389f0721a65&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:12:05.573048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e397a-9dcd-45f6-b540-89abf4c4d165 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:05.573643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e397a-9dcd-45f6-b540-89abf4c4d165 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:05.573829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e397a-9dcd-45f6-b540-89abf4c4d165 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:05.575298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 620/1261] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:12:05.582591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c587d37-6ea1-40d7-b318-5ef2e85fc57c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:05.583154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c587d37-6ea1-40d7-b318-5ef2e85fc57c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:05.583275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c587d37-6ea1-40d7-b318-5ef2e85fc57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:05.602067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5c587d37-6ea1-40d7-b318-5ef2e85fc57c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:12:05.604729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5c587d37-6ea1-40d7-b318-5ef2e85fc57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:12:05.709959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 642/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:12:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:12:05.739235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e397a-9dcd-45f6-b540-89abf4c4d165 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:05.756429 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:05.887129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e397a-9dcd-45f6-b540-89abf4c4d165 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:05.929005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ce9fb-8753-4314-8e2f-85364a2fca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:05.929850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ce9fb-8753-4314-8e2f-85364a2fca65 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:05.930137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ce9fb-8753-4314-8e2f-85364a2fca65 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:05.930419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ce9fb-8753-4314-8e2f-85364a2fca65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:05.943247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ce9fb-8753-4314-8e2f-85364a2fca65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-889ce9fb-8753-4314-8e2f-85364a2fca65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:12:05.951314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:05.953434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:05.955237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 621/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:12:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:05.959725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c587d37-6ea1-40d7-b318-5ef2e85fc57c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:05.989768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:06.112526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c587d37-6ea1-40d7-b318-5ef2e85fc57c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:06.155042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f063694-1f03-4bdb-84fb-c9e0c578ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:06.155769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f063694-1f03-4bdb-84fb-c9e0c578ca02 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:06.156148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f063694-1f03-4bdb-84fb-c9e0c578ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:06.156427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f063694-1f03-4bdb-84fb-c9e0c578ca02 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:06.157194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f063694-1f03-4bdb-84fb-c9e0c578ca02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:06.157500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f063694-1f03-4bdb-84fb-c9e0c578ca02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:06.158951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f063694-1f03-4bdb-84fb-c9e0c578ca02 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:06.169633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 643/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:12:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:06.176806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ce9fb-8753-4314-8e2f-85364a2fca65 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:06.412803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:06.465400 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ce9fb-8753-4314-8e2f-85364a2fca65 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:06.512750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b39f8-0fd6-4db3-8284-17e55cf4d0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:06.513599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b39f8-0fd6-4db3-8284-17e55cf4d0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:06.513914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b39f8-0fd6-4db3-8284-17e55cf4d0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:06.514272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b39f8-0fd6-4db3-8284-17e55cf4d0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:06.528019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b39f8-0fd6-4db3-8284-17e55cf4d0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:06.529698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b39f8-0fd6-4db3-8284-17e55cf4d0a7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-869b39f8-0fd6-4db3-8284-17e55cf4d0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:12:06.552205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:06.553201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:06.554440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:12:06.554440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:12:06.554440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:06.554440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:06.554440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 622/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:06.560520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f063694-1f03-4bdb-84fb-c9e0c578ca02 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:06.656392 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:06.680108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f063694-1f03-4bdb-84fb-c9e0c578ca02 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:06.723934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c169f71-a0d6-459c-adfc-d4af25b576c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:06.724621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c169f71-a0d6-459c-adfc-d4af25b576c5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:06.724909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c169f71-a0d6-459c-adfc-d4af25b576c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:06.725198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c169f71-a0d6-459c-adfc-d4af25b576c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:06.726207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c169f71-a0d6-459c-adfc-d4af25b576c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:06.726551 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c169f71-a0d6-459c-adfc-d4af25b576c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:06.728348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c169f71-a0d6-459c-adfc-d4af25b576c5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:06.745528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 644/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:06.751944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b39f8-0fd6-4db3-8284-17e55cf4d0a7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:06.865460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b39f8-0fd6-4db3-8284-17e55cf4d0a7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:06.888342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:06.904685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb3a9c-772c-4fbb-b23a-b50082827f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:06.905556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb3a9c-772c-4fbb-b23a-b50082827f2a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:06.906016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb3a9c-772c-4fbb-b23a-b50082827f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:06.906322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb3a9c-772c-4fbb-b23a-b50082827f2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:06.915549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb3a9c-772c-4fbb-b23a-b50082827f2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:06.917205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb3a9c-772c-4fbb-b23a-b50082827f2a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-38eb3a9c-772c-4fbb-b23a-b50082827f2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:12:06.934333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:06.935734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:06.936946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 623/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:12:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:12:06.942460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c169f71-a0d6-459c-adfc-d4af25b576c5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:07.062169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c169f71-a0d6-459c-adfc-d4af25b576c5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:07.102763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4789f-398b-4be6-ae46-4ef6470000ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:07.103448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4789f-398b-4be6-ae46-4ef6470000ab None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:07.103734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4789f-398b-4be6-ae46-4ef6470000ab None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:07.104014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4789f-398b-4be6-ae46-4ef6470000ab None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:07.104863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4789f-398b-4be6-ae46-4ef6470000ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:07.105174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4789f-398b-4be6-ae46-4ef6470000ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:07.106661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4789f-398b-4be6-ae46-4ef6470000ab None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:07.117494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:07.127637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 645/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 6 14:12:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:12:07.134028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb3a9c-772c-4fbb-b23a-b50082827f2a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:07.247531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb3a9c-772c-4fbb-b23a-b50082827f2a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:07.286133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433242c-08a2-4f06-bc86-5f6a300504e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:07.286924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433242c-08a2-4f06-bc86-5f6a300504e1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:07.287207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433242c-08a2-4f06-bc86-5f6a300504e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:07.287480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433242c-08a2-4f06-bc86-5f6a300504e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:07.298545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433242c-08a2-4f06-bc86-5f6a300504e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:07.301160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433242c-08a2-4f06-bc86-5f6a300504e1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9433242c-08a2-4f06-bc86-5f6a300504e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:12:07.320575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:07.322374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:07.324250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:12:07.324250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:12:07.324250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:07.324250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:07.324250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 624/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:07.330718 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4789f-398b-4be6-ae46-4ef6470000ab None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:07.444283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4789f-398b-4be6-ae46-4ef6470000ab None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:07.483810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e74fcc-b3da-478f-bd8d-c040657dccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:07.484511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e74fcc-b3da-478f-bd8d-c040657dccd8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:07.484801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e74fcc-b3da-478f-bd8d-c040657dccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:07.485084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e74fcc-b3da-478f-bd8d-c040657dccd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:07.485889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e74fcc-b3da-478f-bd8d-c040657dccd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:07.486226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e74fcc-b3da-478f-bd8d-c040657dccd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:07.488723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e74fcc-b3da-478f-bd8d-c040657dccd8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:07.503026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 646/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:07.508810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433242c-08a2-4f06-bc86-5f6a300504e1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:07.544519 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:07.628052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433242c-08a2-4f06-bc86-5f6a300504e1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:07.666690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a5d5b-217c-4906-b517-02f7bafa9e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:07.667548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a5d5b-217c-4906-b517-02f7bafa9e92 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:07.667887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a5d5b-217c-4906-b517-02f7bafa9e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:07.668150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a5d5b-217c-4906-b517-02f7bafa9e92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:07.676250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a5d5b-217c-4906-b517-02f7bafa9e92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:07.677759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a5d5b-217c-4906-b517-02f7bafa9e92 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-115a5d5b-217c-4906-b517-02f7bafa9e92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:12:07.694113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:07.696092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:07.698215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:12:07.698215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:12:07.698215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:12:07.698215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:12:07.698215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:12:07.698215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:12:07.698215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:12:07.698215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:07.698215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 625/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:12:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:07.703285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e74fcc-b3da-478f-bd8d-c040657dccd8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:07.781095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:07.832415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e74fcc-b3da-478f-bd8d-c040657dccd8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:07.871518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b650878-c810-4fd7-9461-7cee875ae65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:07.872339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b650878-c810-4fd7-9461-7cee875ae65b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:07.872700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b650878-c810-4fd7-9461-7cee875ae65b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:07.873057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b650878-c810-4fd7-9461-7cee875ae65b None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:07.874128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b650878-c810-4fd7-9461-7cee875ae65b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:07.874524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b650878-c810-4fd7-9461-7cee875ae65b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:07.876160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b650878-c810-4fd7-9461-7cee875ae65b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:07.891761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 647/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:12:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:07.899096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a5d5b-217c-4906-b517-02f7bafa9e92 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:08.003455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:08.013885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a5d5b-217c-4906-b517-02f7bafa9e92 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:08.055144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44095804-75b8-4619-8aaa-f4ea25a546e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:08.056104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44095804-75b8-4619-8aaa-f4ea25a546e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:08.056499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44095804-75b8-4619-8aaa-f4ea25a546e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:08.056887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44095804-75b8-4619-8aaa-f4ea25a546e4 None admin] PATH_INFO: `/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/51b91456a0634e0fa22fb389f0721a65/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:08.059358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44095804-75b8-4619-8aaa-f4ea25a546e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2adb074ba2944fb82d24492b94b012b, user_id=51b91456a0634e0fa22fb389f0721a65, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:08.060969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44095804-75b8-4619-8aaa-f4ea25a546e4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:08.076873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 626/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 6 14:12:07 2022] PUT /identity/v3/projects/f2adb074ba2944fb82d24492b94b012b/users/51b91456a0634e0fa22fb389f0721a65/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:12:10.530146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6e6aa-27a5-46ac-b194-c888ce46ade8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:10.530823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6e6aa-27a5-46ac-b194-c888ce46ade8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:10.530987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6e6aa-27a5-46ac-b194-c888ce46ade8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:10.532366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 648/1274] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:12:10.539879 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d43b0-80b1-46a5-8708-8eedfc26ea20 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:10.540495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d43b0-80b1-46a5-8708-8eedfc26ea20 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:10.540495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d43b0-80b1-46a5-8708-8eedfc26ea20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:10.559494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-328d43b0-80b1-46a5-8708-8eedfc26ea20 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:12:10.560939 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-328d43b0-80b1-46a5-8708-8eedfc26ea20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:12:10.643948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 627/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:12:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:12:10.671831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6e6aa-27a5-46ac-b194-c888ce46ade8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:10.829010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6e6aa-27a5-46ac-b194-c888ce46ade8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:10.847525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:10.868996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b675478-a91d-4bfe-9b79-87237b7472f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:10.869956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b675478-a91d-4bfe-9b79-87237b7472f1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:10.870278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b675478-a91d-4bfe-9b79-87237b7472f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:10.870591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b675478-a91d-4bfe-9b79-87237b7472f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:10.883271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b675478-a91d-4bfe-9b79-87237b7472f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-6b675478-a91d-4bfe-9b79-87237b7472f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 14:12:10.891666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:10.893031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:10.895052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 649/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 6 14:12:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:10.900229 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d43b0-80b1-46a5-8708-8eedfc26ea20 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:10.988077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:11.009892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d43b0-80b1-46a5-8708-8eedfc26ea20 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:11.047471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050a75-a5f2-4ae0-8887-7438c678893e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:11.048134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050a75-a5f2-4ae0-8887-7438c678893e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:11.048419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050a75-a5f2-4ae0-8887-7438c678893e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:11.048697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050a75-a5f2-4ae0-8887-7438c678893e None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:11.049449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050a75-a5f2-4ae0-8887-7438c678893e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:11.049803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050a75-a5f2-4ae0-8887-7438c678893e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:11.051739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050a75-a5f2-4ae0-8887-7438c678893e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:11.061185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 628/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 6 14:12:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:11.068469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b675478-a91d-4bfe-9b79-87237b7472f1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:11.201570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b675478-a91d-4bfe-9b79-87237b7472f1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:11.243391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4734a01-2496-4820-9771-47d5bc750bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:11.244281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4734a01-2496-4820-9771-47d5bc750bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:11.244556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4734a01-2496-4820-9771-47d5bc750bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:11.244835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4734a01-2496-4820-9771-47d5bc750bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:11.258276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4734a01-2496-4820-9771-47d5bc750bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:11.259777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4734a01-2496-4820-9771-47d5bc750bc5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a4734a01-2496-4820-9771-47d5bc750bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:12:11.278878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:11.280083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:11.281288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:12:11.281288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:12:11.281288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:11.281288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:11.281871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 650/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:12:11.286933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050a75-a5f2-4ae0-8887-7438c678893e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:11.412274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050a75-a5f2-4ae0-8887-7438c678893e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:11.451101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454111-8984-4c01-8200-e5aed45d9095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:11.451859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454111-8984-4c01-8200-e5aed45d9095 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:11.452086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454111-8984-4c01-8200-e5aed45d9095 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:11.452361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454111-8984-4c01-8200-e5aed45d9095 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:11.453110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31454111-8984-4c01-8200-e5aed45d9095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:11.453416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31454111-8984-4c01-8200-e5aed45d9095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:11.454884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31454111-8984-4c01-8200-e5aed45d9095 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:11.469605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 629/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:11.476822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4734a01-2496-4820-9771-47d5bc750bc5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:11.515175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:11.602948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4734a01-2496-4820-9771-47d5bc750bc5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:11.642034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d304658-4ff7-4c9a-93dc-39e32eac737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:11.643079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d304658-4ff7-4c9a-93dc-39e32eac737e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:11.643513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d304658-4ff7-4c9a-93dc-39e32eac737e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:11.643951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d304658-4ff7-4c9a-93dc-39e32eac737e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:11.659195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d304658-4ff7-4c9a-93dc-39e32eac737e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:11.662790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d304658-4ff7-4c9a-93dc-39e32eac737e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1d304658-4ff7-4c9a-93dc-39e32eac737e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 14:12:11.682766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:11.689299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:11.690601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 651/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:12:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:11.691320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454111-8984-4c01-8200-e5aed45d9095 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:11.695281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:11.807163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454111-8984-4c01-8200-e5aed45d9095 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:11.851986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13545d-8644-4335-a8f9-81dfa2dbcdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:11.852682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13545d-8644-4335-a8f9-81dfa2dbcdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:11.852982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13545d-8644-4335-a8f9-81dfa2dbcdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:11.853296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13545d-8644-4335-a8f9-81dfa2dbcdf2 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:11.854240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13545d-8644-4335-a8f9-81dfa2dbcdf2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:11.854568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13545d-8644-4335-a8f9-81dfa2dbcdf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:11.856103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13545d-8644-4335-a8f9-81dfa2dbcdf2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:11.875574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 630/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 6 14:12:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:12:11.882309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d304658-4ff7-4c9a-93dc-39e32eac737e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:11.990623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:12.003636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d304658-4ff7-4c9a-93dc-39e32eac737e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:12.042841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b103a78-6b37-4ee9-ac2f-26ce88cb1e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:12.043569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b103a78-6b37-4ee9-ac2f-26ce88cb1e5d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:12.043856 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b103a78-6b37-4ee9-ac2f-26ce88cb1e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:12.044130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b103a78-6b37-4ee9-ac2f-26ce88cb1e5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:12.124600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b103a78-6b37-4ee9-ac2f-26ce88cb1e5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:12.126103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b103a78-6b37-4ee9-ac2f-26ce88cb1e5d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:12.225680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:12.237262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 652/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 6 14:12:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 14:12:12.243316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13545d-8644-4335-a8f9-81dfa2dbcdf2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:12.311173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:12.355156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13545d-8644-4335-a8f9-81dfa2dbcdf2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:12.392549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3370c0-b1a0-4859-9254-6daa7c75a257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:12.393256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3370c0-b1a0-4859-9254-6daa7c75a257 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:12.393538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3370c0-b1a0-4859-9254-6daa7c75a257 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:12.394897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3370c0-b1a0-4859-9254-6daa7c75a257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:12.406283 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3370c0-b1a0-4859-9254-6daa7c75a257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:12.407870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3370c0-b1a0-4859-9254-6daa7c75a257 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ec3370c0-b1a0-4859-9254-6daa7c75a257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 14:12:12.427075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:12.429149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:12.432398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 14:12:12.432398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 14:12:12.432398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 14:12:12.432398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:12.432398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 631/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:12.435118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b103a78-6b37-4ee9-ac2f-26ce88cb1e5d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:12.561415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b103a78-6b37-4ee9-ac2f-26ce88cb1e5d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:12.602918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcdae1-0895-4e0e-9b40-81b577bc1472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:12.603620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcdae1-0895-4e0e-9b40-81b577bc1472 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:12.604204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcdae1-0895-4e0e-9b40-81b577bc1472 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:12.604204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcdae1-0895-4e0e-9b40-81b577bc1472 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:12.604921 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcdae1-0895-4e0e-9b40-81b577bc1472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:12.605230 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcdae1-0895-4e0e-9b40-81b577bc1472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:12.606756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcdae1-0895-4e0e-9b40-81b577bc1472 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:12.620665 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 653/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 6 14:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:12.627244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3370c0-b1a0-4859-9254-6daa7c75a257 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:12.746545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3370c0-b1a0-4859-9254-6daa7c75a257 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:12.762678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:12.785364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75ab13-1f6d-4b3d-ab7a-85f03a84756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:12.786204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75ab13-1f6d-4b3d-ab7a-85f03a84756f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:12.786486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75ab13-1f6d-4b3d-ab7a-85f03a84756f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:12.786848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75ab13-1f6d-4b3d-ab7a-85f03a84756f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:12.795114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75ab13-1f6d-4b3d-ab7a-85f03a84756f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:12.796556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75ab13-1f6d-4b3d-ab7a-85f03a84756f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-cb75ab13-1f6d-4b3d-ab7a-85f03a84756f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 14:12:12.811495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:12.812631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:12.813675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 14:12:12.813675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 14:12:12.813675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 14:12:12.813675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 14:12:12.813675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 14:12:12.813675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 14:12:12.813675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 14:12:12.813675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jun 06 14:12:12.813675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 632/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 6 14:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:12.819916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcdae1-0895-4e0e-9b40-81b577bc1472 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:12.878495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:12.932713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcdae1-0895-4e0e-9b40-81b577bc1472 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:12.969006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5f1d3-1230-4137-a6fa-ebab0bc73c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:12.969740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5f1d3-1230-4137-a6fa-ebab0bc73c04 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:12.970020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5f1d3-1230-4137-a6fa-ebab0bc73c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:12.970293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5f1d3-1230-4137-a6fa-ebab0bc73c04 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:12.971082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5f1d3-1230-4137-a6fa-ebab0bc73c04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:12.971410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5f1d3-1230-4137-a6fa-ebab0bc73c04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:12.973151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5f1d3-1230-4137-a6fa-ebab0bc73c04 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:12.985610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 654/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 6 14:12:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:12.995052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75ab13-1f6d-4b3d-ab7a-85f03a84756f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:13.107968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75ab13-1f6d-4b3d-ab7a-85f03a84756f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:13.147015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550b202-21c7-4dc4-9724-a2db03f90127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:13.147864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550b202-21c7-4dc4-9724-a2db03f90127 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:13.147864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550b202-21c7-4dc4-9724-a2db03f90127 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:13.148512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550b202-21c7-4dc4-9724-a2db03f90127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:13.148859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550b202-21c7-4dc4-9724-a2db03f90127 None admin] RBAC: Adding query filter params (role.id=d6ad1e18a93d4557a8581218e0802d90, scope.project.id=f2adb074ba2944fb82d24492b94b012b, user.id=51b91456a0634e0fa22fb389f0721a65) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:13.149216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550b202-21c7-4dc4-9724-a2db03f90127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:13.150715 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550b202-21c7-4dc4-9724-a2db03f90127 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:13.157268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 633/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 6 14:12:12 2022] GET /identity/v3/role_assignments?user.id=51b91456a0634e0fa22fb389f0721a65&scope.project.id=f2adb074ba2944fb82d24492b94b012b&role.id=d6ad1e18a93d4557a8581218e0802d90 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:15.590818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d86f-75b5-4c82-b263-24d8fd236ed7 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:15.591388 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d86f-75b5-4c82-b263-24d8fd236ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:15.591388 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d86f-75b5-4c82-b263-24d8fd236ed7 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:15.592741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 655/1288] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:12:15.598629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7d92f-c2c2-46c8-92ef-8621b89eae59 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:15.599156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7d92f-c2c2-46c8-92ef-8621b89eae59 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:15.599156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7d92f-c2c2-46c8-92ef-8621b89eae59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:15.618619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-98e7d92f-c2c2-46c8-92ef-8621b89eae59 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:12:15.619916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-98e7d92f-c2c2-46c8-92ef-8621b89eae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:12:15.656393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:15.699695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 634/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:12:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:12:15.725003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d86f-75b5-4c82-b263-24d8fd236ed7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:15.841023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d86f-75b5-4c82-b263-24d8fd236ed7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:15.853050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:15.881271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57ce68-1b86-42c8-ab72-2e9f808fbcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:15.882114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57ce68-1b86-42c8-ab72-2e9f808fbcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:15.882397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57ce68-1b86-42c8-ab72-2e9f808fbcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:15.882671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57ce68-1b86-42c8-ab72-2e9f808fbcf1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:15.883386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57ce68-1b86-42c8-ab72-2e9f808fbcf1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:15.884933 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57ce68-1b86-42c8-ab72-2e9f808fbcf1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-ad57ce68-1b86-42c8-ab72-2e9f808fbcf1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:15.898394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:15.905208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:15.906296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 656/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:12:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 14:12:15.907162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7d92f-c2c2-46c8-92ef-8621b89eae59 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:16.016375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7d92f-c2c2-46c8-92ef-8621b89eae59 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:16.053247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be97ca-bb35-45fb-926e-1873ca4834a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:16.053943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be97ca-bb35-45fb-926e-1873ca4834a0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:16.054230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be97ca-bb35-45fb-926e-1873ca4834a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:16.054510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be97ca-bb35-45fb-926e-1873ca4834a0 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:16.055223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be97ca-bb35-45fb-926e-1873ca4834a0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:16.055526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be97ca-bb35-45fb-926e-1873ca4834a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:16.056891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be97ca-bb35-45fb-926e-1873ca4834a0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:16.064224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 635/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Jun 6 14:12:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:16.070744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57ce68-1b86-42c8-ab72-2e9f808fbcf1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:16.183364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57ce68-1b86-42c8-ab72-2e9f808fbcf1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:16.228324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769ce66-0ccb-4b62-9f71-41013da6f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:16.229013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ce66-0ccb-4b62-9f71-41013da6f1cb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:16.229302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ce66-0ccb-4b62-9f71-41013da6f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:16.229583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ce66-0ccb-4b62-9f71-41013da6f1cb None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:16.230347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769ce66-0ccb-4b62-9f71-41013da6f1cb None admin] RBAC: Adding query filter params (type=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:16.230656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769ce66-0ccb-4b62-9f71-41013da6f1cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:16.232104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769ce66-0ccb-4b62-9f71-41013da6f1cb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:16.237934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 657/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Jun 6 14:12:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:12:18.587447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28c433-403b-453b-87b6-b8054b1128a1 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:18.587983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28c433-403b-453b-87b6-b8054b1128a1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:18.587983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28c433-403b-453b-87b6-b8054b1128a1 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:18.589358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 636/1293] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:12:18.594834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9280-94e6-47b5-b089-359d0ab51359 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:18.595325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9280-94e6-47b5-b089-359d0ab51359 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:18.595432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9280-94e6-47b5-b089-359d0ab51359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:18.614799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-946e9280-94e6-47b5-b089-359d0ab51359 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:12:18.616001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-946e9280-94e6-47b5-b089-359d0ab51359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:12:18.650987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:18.696839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 658/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:12:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:12:18.722067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28c433-403b-453b-87b6-b8054b1128a1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:18.763051 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:18.854810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28c433-403b-453b-87b6-b8054b1128a1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:18.893722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3207f7-6631-4fe3-86ae-8ae161751bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:18.894482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3207f7-6631-4fe3-86ae-8ae161751bfd None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:18.894802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3207f7-6631-4fe3-86ae-8ae161751bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:18.895320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3207f7-6631-4fe3-86ae-8ae161751bfd None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:18.896064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3207f7-6631-4fe3-86ae-8ae161751bfd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:18.897584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3207f7-6631-4fe3-86ae-8ae161751bfd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:18.914122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 637/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 6 14:12:18 2022] POST /identity/v3/services => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:12:21.509800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f43ea-a47a-47e3-9c0d-ec88b174a027 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:21.510359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f43ea-a47a-47e3-9c0d-ec88b174a027 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:21.510359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f43ea-a47a-47e3-9c0d-ec88b174a027 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:21.511747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 659/1296] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:12:21.518914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12477c7-557f-40f4-bbef-0ab7b4ff0bb1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:21.519403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12477c7-557f-40f4-bbef-0ab7b4ff0bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:21.519554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12477c7-557f-40f4-bbef-0ab7b4ff0bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:21.539096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e12477c7-557f-40f4-bbef-0ab7b4ff0bb1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:12:21.540470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e12477c7-557f-40f4-bbef-0ab7b4ff0bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:12:21.622570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 638/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:12:21 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:12:21.648505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f43ea-a47a-47e3-9c0d-ec88b174a027 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:21.781244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f43ea-a47a-47e3-9c0d-ec88b174a027 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:21.820768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5d5d6-2aaf-482b-916c-f234b64a607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:21.821620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5d5d6-2aaf-482b-916c-f234b64a607e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:21.821954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5d5d6-2aaf-482b-916c-f234b64a607e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:21.822235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5d5d6-2aaf-482b-916c-f234b64a607e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:21.822975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5d5d6-2aaf-482b-916c-f234b64a607e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:21.824470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5d5d6-2aaf-482b-916c-f234b64a607e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-23a5d5d6-2aaf-482b-916c-f234b64a607e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:21.838371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:21.839623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:21.840600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 660/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:12:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:21.849472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12477c7-557f-40f4-bbef-0ab7b4ff0bb1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:21.872714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:21.977571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12477c7-557f-40f4-bbef-0ab7b4ff0bb1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:22.014648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce3ce2-d6a1-44ea-8a7a-a800537e8c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:22.015307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce3ce2-d6a1-44ea-8a7a-a800537e8c00 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:22.015591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce3ce2-d6a1-44ea-8a7a-a800537e8c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:22.015867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce3ce2-d6a1-44ea-8a7a-a800537e8c00 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:22.016579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce3ce2-d6a1-44ea-8a7a-a800537e8c00 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:22.016882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce3ce2-d6a1-44ea-8a7a-a800537e8c00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:22.018291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce3ce2-d6a1-44ea-8a7a-a800537e8c00 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:22.024437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 639/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Jun 6 14:12:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:22.031509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5d5d6-2aaf-482b-916c-f234b64a607e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:22.069729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:22.151788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5d5d6-2aaf-482b-916c-f234b64a607e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:22.193673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b501395-f3ed-4e2c-8cde-650b32d2b350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:22.194508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b501395-f3ed-4e2c-8cde-650b32d2b350 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:22.194796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b501395-f3ed-4e2c-8cde-650b32d2b350 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:22.195175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b501395-f3ed-4e2c-8cde-650b32d2b350 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:22.196130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b501395-f3ed-4e2c-8cde-650b32d2b350 None admin] RBAC: Adding query filter params (interface=public, service_id=bfc05784fcb64d128b51df9d06445579, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:22.196443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b501395-f3ed-4e2c-8cde-650b32d2b350 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:22.198033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b501395-f3ed-4e2c-8cde-650b32d2b350 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:22.203895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 661/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 6 14:12:22 2022] GET /identity/v3/endpoints?service_id=bfc05784fcb64d128b51df9d06445579&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:22.209849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce3ce2-d6a1-44ea-8a7a-a800537e8c00 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:22.299482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:22.320498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce3ce2-d6a1-44ea-8a7a-a800537e8c00 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:22.356817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced57a8-505a-413d-958f-330faf11f2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:22.357516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced57a8-505a-413d-958f-330faf11f2cb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:22.357872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced57a8-505a-413d-958f-330faf11f2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:22.358186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced57a8-505a-413d-958f-330faf11f2cb None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:22.358958 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced57a8-505a-413d-958f-330faf11f2cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:22.360457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced57a8-505a-413d-958f-330faf11f2cb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:22.370309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 640/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 6 14:12:22 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:12:24.925097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0208b9b-9f13-43f5-87bc-196432a23338 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:24.925740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0208b9b-9f13-43f5-87bc-196432a23338 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:24.925740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0208b9b-9f13-43f5-87bc-196432a23338 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:24.926983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 662/1302] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:12:24.932686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded143be-5f56-4c0a-9671-75dcc4687dc7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:24.933147 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded143be-5f56-4c0a-9671-75dcc4687dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:24.933273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded143be-5f56-4c0a-9671-75dcc4687dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:24.951884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ded143be-5f56-4c0a-9671-75dcc4687dc7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:12:24.953053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ded143be-5f56-4c0a-9671-75dcc4687dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:12:25.038011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 641/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:12:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 14:12:25.066123 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0208b9b-9f13-43f5-87bc-196432a23338 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:25.215064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:25.224831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0208b9b-9f13-43f5-87bc-196432a23338 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:25.260882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7340a2-743d-41a8-84d0-e5c092d20017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:25.261945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7340a2-743d-41a8-84d0-e5c092d20017 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:25.262236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7340a2-743d-41a8-84d0-e5c092d20017 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:25.262541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7340a2-743d-41a8-84d0-e5c092d20017 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:25.263323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7340a2-743d-41a8-84d0-e5c092d20017 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:25.264749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7340a2-743d-41a8-84d0-e5c092d20017 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-3a7340a2-743d-41a8-84d0-e5c092d20017 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 14:12:25.279797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 14:12:25.281030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Jun 06 14:12:25.282308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 663/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 6 14:12:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 14:12:25.290467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded143be-5f56-4c0a-9671-75dcc4687dc7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:25.394783 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:25.419550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded143be-5f56-4c0a-9671-75dcc4687dc7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:25.457874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce3ebd-a652-4920-8034-80fe4da19aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:25.458594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce3ebd-a652-4920-8034-80fe4da19aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:25.458902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce3ebd-a652-4920-8034-80fe4da19aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:25.459201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce3ebd-a652-4920-8034-80fe4da19aa2 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:25.459978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce3ebd-a652-4920-8034-80fe4da19aa2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:12:25.460314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce3ebd-a652-4920-8034-80fe4da19aa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:25.461692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce3ebd-a652-4920-8034-80fe4da19aa2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:25.467290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 642/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Jun 6 14:12:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:12:25.473950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7340a2-743d-41a8-84d0-e5c092d20017 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:12:25.580005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7340a2-743d-41a8-84d0-e5c092d20017 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:12:25.616468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11844752-e641-4665-8211-053a09cab87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:12:25.617134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11844752-e641-4665-8211-053a09cab87b None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:12:25.617420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11844752-e641-4665-8211-053a09cab87b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:12:25.617699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11844752-e641-4665-8211-053a09cab87b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:12:25.618491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11844752-e641-4665-8211-053a09cab87b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:12:25.619895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11844752-e641-4665-8211-053a09cab87b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:12:25.641898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 664/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 6 14:12:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:12:35.464406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=30 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:12:35.636835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:34.615037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9ee8d-00ca-42ec-a8e7-17dc76ed2c0f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:34.615508 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9ee8d-00ca-42ec-a8e7-17dc76ed2c0f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:34.616069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9ee8d-00ca-42ec-a8e7-17dc76ed2c0f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:34.618430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 643/1307] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:13:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:34.627860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6e990-bbd8-42c3-a1ff-f91b5ee3d7c1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:34.628583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6e990-bbd8-42c3-a1ff-f91b5ee3d7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:34.629107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6e990-bbd8-42c3-a1ff-f91b5ee3d7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:34.812866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a0e6e990-bbd8-42c3-a1ff-f91b5ee3d7c1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:34.814333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a0e6e990-bbd8-42c3-a1ff-f91b5ee3d7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:35.059020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 665/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:13:35.108264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b6ee9-2618-4aab-b577-354e8c916af3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:35.113450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b6ee9-2618-4aab-b577-354e8c916af3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:35.113921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b6ee9-2618-4aab-b577-354e8c916af3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:35.116178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 644/1309] 10.222.0.13 () {54 vars in 964 bytes} [Mon Jun 6 14:13:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:35.122658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcac80a-d173-489f-a245-77a9b949cae1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:35.123092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcac80a-d173-489f-a245-77a9b949cae1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:35.123526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcac80a-d173-489f-a245-77a9b949cae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:35.210872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1fcac80a-d173-489f-a245-77a9b949cae1 None None] MFA Rules not processed for user `59cd9d922204496d8ea185716b832481`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:35.212370 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1fcac80a-d173-489f-a245-77a9b949cae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:35.235740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:35.311837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b6ee9-2618-4aab-b577-354e8c916af3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:13:35.314005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-446b6ee9-2618-4aab-b577-354e8c916af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:35.318483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 666/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Jun 6 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:13:35.460148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b6ee9-2618-4aab-b577-354e8c916af3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:13:35.552045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ec427-22ce-4caf-bd14-e47512846de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:13:35.552900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ec427-22ce-4caf-bd14-e47512846de3 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:35.553185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ec427-22ce-4caf-bd14-e47512846de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:35.553463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ec427-22ce-4caf-bd14-e47512846de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:35.555183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-253ec427-22ce-4caf-bd14-e47512846de3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:35.620475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:35.684437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ec427-22ce-4caf-bd14-e47512846de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:13:35.685920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ec427-22ce-4caf-bd14-e47512846de3 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:13:35.709026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 645/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:13:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:13:38.384834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c6f73-e3c0-4430-938b-d0ab1f60d8c5 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:38.385394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c6f73-e3c0-4430-938b-d0ab1f60d8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:38.385394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c6f73-e3c0-4430-938b-d0ab1f60d8c5 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:38.386817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 667/1312] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:38.392971 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de3235-989c-4a04-a046-bf5f814388b0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:38.393536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de3235-989c-4a04-a046-bf5f814388b0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:38.393536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de3235-989c-4a04-a046-bf5f814388b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:38.486799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b5de3235-989c-4a04-a046-bf5f814388b0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:38.488011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b5de3235-989c-4a04-a046-bf5f814388b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:38.608237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 646/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:13:38.651097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c6f73-e3c0-4430-938b-d0ab1f60d8c5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:13:38.652620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-203c6f73-e3c0-4430-938b-d0ab1f60d8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:38.894695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c6f73-e3c0-4430-938b-d0ab1f60d8c5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:13:38.988173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:38.991238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa1cc-2d61-4606-961e-2fa379f5fd1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:13:38.991948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa1cc-2d61-4606-961e-2fa379f5fd1c service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:38.992387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa1cc-2d61-4606-961e-2fa379f5fd1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:38.992708 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa1cc-2d61-4606-961e-2fa379f5fd1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:38.994639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-205aa1cc-2d61-4606-961e-2fa379f5fd1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:39.107603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205aa1cc-2d61-4606-961e-2fa379f5fd1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:13:39.109119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205aa1cc-2d61-4606-961e-2fa379f5fd1c service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:13:39.128023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 668/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:13:41.802963 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba156cb3-59ec-4fe4-8c2b-1c40709b6b93 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:41.803554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba156cb3-59ec-4fe4-8c2b-1c40709b6b93 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:41.803554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba156cb3-59ec-4fe4-8c2b-1c40709b6b93 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:41.804865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 647/1315] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:41.810324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7b45d-2ba3-4265-a6cd-067c0d4214f4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:41.811279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7b45d-2ba3-4265-a6cd-067c0d4214f4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:41.811413 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7b45d-2ba3-4265-a6cd-067c0d4214f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:41.877559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-62f7b45d-2ba3-4265-a6cd-067c0d4214f4 None None] MFA Rules not processed for user `0e961d4918e44e499e44d50410595f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:41.878824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-62f7b45d-2ba3-4265-a6cd-067c0d4214f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:41.932448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:41.984904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 669/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:13:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:13:42.041386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006750b-4288-4a70-9720-1fc29cf4825f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:42.041675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006750b-4288-4a70-9720-1fc29cf4825f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:42.041991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006750b-4288-4a70-9720-1fc29cf4825f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:42.043411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 648/1317] 10.222.0.13 () {54 vars in 964 bytes} [Mon Jun 6 14:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:42.050692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13ca97-4da9-4b5c-b6b8-7cbfa9750e57 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:42.050980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13ca97-4da9-4b5c-b6b8-7cbfa9750e57 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:42.051254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13ca97-4da9-4b5c-b6b8-7cbfa9750e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:42.131690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cf13ca97-4da9-4b5c-b6b8-7cbfa9750e57 None None] MFA Rules not processed for user `59cd9d922204496d8ea185716b832481`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:42.132806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cf13ca97-4da9-4b5c-b6b8-7cbfa9750e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:42.209544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 670/1318] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Jun 6 14:13:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:13:42.218917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006750b-4288-4a70-9720-1fc29cf4825f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:13:42.220440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4006750b-4288-4a70-9720-1fc29cf4825f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:42.230404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:42.345149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006750b-4288-4a70-9720-1fc29cf4825f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:13:42.423475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76078b-b465-404d-805b-ceffa1a3cfa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:13:42.424029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76078b-b465-404d-805b-ceffa1a3cfa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:42.424314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76078b-b465-404d-805b-ceffa1a3cfa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:42.424591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76078b-b465-404d-805b-ceffa1a3cfa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:42.426284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cc76078b-b465-404d-805b-ceffa1a3cfa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:42.491881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:42.542195 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76078b-b465-404d-805b-ceffa1a3cfa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:13:42.543634 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76078b-b465-404d-805b-ceffa1a3cfa2 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:13:42.564301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 649/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:13:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:13:45.858394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54514418-0004-4755-9d75-f798628292e7 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:45.858931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54514418-0004-4755-9d75-f798628292e7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:45.858931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54514418-0004-4755-9d75-f798628292e7 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:45.860468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 671/1320] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:13:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:45.865974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015092ef-040b-47ff-8e2c-1072c2e40521 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:45.866401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015092ef-040b-47ff-8e2c-1072c2e40521 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:45.866510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015092ef-040b-47ff-8e2c-1072c2e40521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:45.931338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-015092ef-040b-47ff-8e2c-1072c2e40521 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:45.932535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-015092ef-040b-47ff-8e2c-1072c2e40521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:46.028965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 650/1321] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:13:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:13:46.068251 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54514418-0004-4755-9d75-f798628292e7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:13:46.069827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-54514418-0004-4755-9d75-f798628292e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:46.127293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:46.183930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54514418-0004-4755-9d75-f798628292e7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:13:46.269916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d309b30-5869-4aa7-9a39-36cb3fe467fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:13:46.270568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d309b30-5869-4aa7-9a39-36cb3fe467fe service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:46.270846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d309b30-5869-4aa7-9a39-36cb3fe467fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:46.271122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d309b30-5869-4aa7-9a39-36cb3fe467fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:46.272775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9d309b30-5869-4aa7-9a39-36cb3fe467fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:46.372916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:46.386809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d309b30-5869-4aa7-9a39-36cb3fe467fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:13:46.388192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d309b30-5869-4aa7-9a39-36cb3fe467fe service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:13:46.407201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 672/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:13:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:13:49.482644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403e613-bf0e-4fb4-a897-e58f3769e4ae None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:49.483237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403e613-bf0e-4fb4-a897-e58f3769e4ae None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:49.483237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403e613-bf0e-4fb4-a897-e58f3769e4ae None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:49.484529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 651/1323] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:49.492446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de55e11-c019-4a2d-b989-cc586da96e8b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:49.492988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de55e11-c019-4a2d-b989-cc586da96e8b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:49.493293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de55e11-c019-4a2d-b989-cc586da96e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:49.585508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9de55e11-c019-4a2d-b989-cc586da96e8b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:49.586963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9de55e11-c019-4a2d-b989-cc586da96e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:49.718929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 673/1324] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:13:49.760677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403e613-bf0e-4fb4-a897-e58f3769e4ae None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:13:49.762292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9403e613-bf0e-4fb4-a897-e58f3769e4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:49.792789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:49.892937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403e613-bf0e-4fb4-a897-e58f3769e4ae None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:13:49.975042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e335c8-df8c-426e-9aab-2bdbefe5447f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:13:49.975837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e335c8-df8c-426e-9aab-2bdbefe5447f service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:49.976183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e335c8-df8c-426e-9aab-2bdbefe5447f service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:49.976533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e335c8-df8c-426e-9aab-2bdbefe5447f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:49.978297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-01e335c8-df8c-426e-9aab-2bdbefe5447f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:50.063071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:50.108740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e335c8-df8c-426e-9aab-2bdbefe5447f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:13:50.111634 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e335c8-df8c-426e-9aab-2bdbefe5447f service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:13:50.147472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 652/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:13:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:13:52.761389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e31e7-a6b1-4100-baef-58a6574f6bfb None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:52.762306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e31e7-a6b1-4100-baef-58a6574f6bfb None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:52.762306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e31e7-a6b1-4100-baef-58a6574f6bfb None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:52.764306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 674/1326] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:52.770243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71a8f8-e890-4493-aa72-781a852d2234 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:52.771103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71a8f8-e890-4493-aa72-781a852d2234 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:52.771103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71a8f8-e890-4493-aa72-781a852d2234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:52.874698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-be71a8f8-e890-4493-aa72-781a852d2234 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:52.875841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-be71a8f8-e890-4493-aa72-781a852d2234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:52.990817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 653/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:13:53.027271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e31e7-a6b1-4100-baef-58a6574f6bfb None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:13:53.029444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3b3e31e7-a6b1-4100-baef-58a6574f6bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:53.113297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:53.164153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e31e7-a6b1-4100-baef-58a6574f6bfb None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:13:53.302719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7e9bc-8d42-476b-9f16-1e0a2efc9a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:13:53.303793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7e9bc-8d42-476b-9f16-1e0a2efc9a95 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:53.304289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7e9bc-8d42-476b-9f16-1e0a2efc9a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:53.304786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7e9bc-8d42-476b-9f16-1e0a2efc9a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:53.308005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-83b7e9bc-8d42-476b-9f16-1e0a2efc9a95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:53.500869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:53.504583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7e9bc-8d42-476b-9f16-1e0a2efc9a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:13:53.507949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7e9bc-8d42-476b-9f16-1e0a2efc9a95 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:13:53.532769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 675/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:13:56.178719 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc97c69-3668-4355-b756-7bd30e8df2a8 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:56.179415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc97c69-3668-4355-b756-7bd30e8df2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:56.179415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc97c69-3668-4355-b756-7bd30e8df2a8 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:56.180682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 654/1329] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:56.188007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f334ae-c6e6-4a86-ba85-e10750ba5cc2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:56.188520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f334ae-c6e6-4a86-ba85-e10750ba5cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:56.188629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f334ae-c6e6-4a86-ba85-e10750ba5cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:56.261499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-46f334ae-c6e6-4a86-ba85-e10750ba5cc2 None None] MFA Rules not processed for user `0e961d4918e44e499e44d50410595f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:56.262906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-46f334ae-c6e6-4a86-ba85-e10750ba5cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:56.378340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 676/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:13:56 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:13:56.418105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc97c69-3668-4355-b756-7bd30e8df2a8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:13:56.420468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-adc97c69-3668-4355-b756-7bd30e8df2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:56.458161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:56.567539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc97c69-3668-4355-b756-7bd30e8df2a8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:13:56.672336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37045ac-6f0f-469e-90ec-51785262fa05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:13:56.673117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37045ac-6f0f-469e-90ec-51785262fa05 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:56.673511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37045ac-6f0f-469e-90ec-51785262fa05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:56.673819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37045ac-6f0f-469e-90ec-51785262fa05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:56.675906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f37045ac-6f0f-469e-90ec-51785262fa05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:13:56.801216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:13:56.866209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37045ac-6f0f-469e-90ec-51785262fa05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:13:56.869025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37045ac-6f0f-469e-90ec-51785262fa05 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:13:56.907403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 655/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:13:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:13:59.859713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccef27-cc46-4c99-add4-98648e8cc048 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:59.860520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccef27-cc46-4c99-add4-98648e8cc048 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:59.860520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccef27-cc46-4c99-add4-98648e8cc048 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:59.861935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 677/1332] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:13:59.867528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffcde6-ea47-49e4-bbaa-a5fd596f9eb3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:13:59.867953 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffcde6-ea47-49e4-bbaa-a5fd596f9eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:13:59.868082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffcde6-ea47-49e4-bbaa-a5fd596f9eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:13:59.942864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-74ffcde6-ea47-49e4-bbaa-a5fd596f9eb3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:13:59.944079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-74ffcde6-ea47-49e4-bbaa-a5fd596f9eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:00.054618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 656/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:00.091580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccef27-cc46-4c99-add4-98648e8cc048 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:00.093190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-82ccef27-cc46-4c99-add4-98648e8cc048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:00.167065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:14:00.216613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccef27-cc46-4c99-add4-98648e8cc048 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:00.308121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e25697-353a-40b9-bfcb-d5c58291bd6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:00.309029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e25697-353a-40b9-bfcb-d5c58291bd6f service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:00.309451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e25697-353a-40b9-bfcb-d5c58291bd6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:00.310489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e25697-353a-40b9-bfcb-d5c58291bd6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:00.313449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-18e25697-353a-40b9-bfcb-d5c58291bd6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:00.498524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e25697-353a-40b9-bfcb-d5c58291bd6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:00.500113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e25697-353a-40b9-bfcb-d5c58291bd6f service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:00.517054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:14:00.528828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 678/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:02.953135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e063593-0a47-4ab5-b1b2-899d23b9671e None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:02.953770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e063593-0a47-4ab5-b1b2-899d23b9671e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:02.953957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e063593-0a47-4ab5-b1b2-899d23b9671e None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:02.955772 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 657/1335] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:14:02.962543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348049d-887f-433b-a13f-b66867c31d46 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:02.963120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348049d-887f-433b-a13f-b66867c31d46 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:02.963546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348049d-887f-433b-a13f-b66867c31d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:03.072083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f348049d-887f-433b-a13f-b66867c31d46 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:03.073312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f348049d-887f-433b-a13f-b66867c31d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:03.181805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 679/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:03.221993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e063593-0a47-4ab5-b1b2-899d23b9671e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:03.223516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3e063593-0a47-4ab5-b1b2-899d23b9671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:03.264875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:14:03.336894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e063593-0a47-4ab5-b1b2-899d23b9671e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:03.414635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a2e23-3db3-4921-a0be-4dd07527e5c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:03.415285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a2e23-3db3-4921-a0be-4dd07527e5c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:03.415564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a2e23-3db3-4921-a0be-4dd07527e5c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:03.415837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a2e23-3db3-4921-a0be-4dd07527e5c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:03.417491 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-786a2e23-3db3-4921-a0be-4dd07527e5c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:03.511959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:14:03.539816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a2e23-3db3-4921-a0be-4dd07527e5c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:03.541288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a2e23-3db3-4921-a0be-4dd07527e5c4 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:03.561979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 658/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:06.392730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b4f03-4979-4eba-90dd-9fd47a2a7e55 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:06.393316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b4f03-4979-4eba-90dd-9fd47a2a7e55 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:06.393316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b4f03-4979-4eba-90dd-9fd47a2a7e55 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:06.394689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 680/1338] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:06.399993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4db3a-91d8-4813-8f48-5a78b08cbb1b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:06.400436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4db3a-91d8-4813-8f48-5a78b08cbb1b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:06.400552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4db3a-91d8-4813-8f48-5a78b08cbb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:06.473203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-aef4db3a-91d8-4813-8f48-5a78b08cbb1b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:06.474481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-aef4db3a-91d8-4813-8f48-5a78b08cbb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:06.586823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 659/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:06.625321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b4f03-4979-4eba-90dd-9fd47a2a7e55 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:06.627465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-819b4f03-4979-4eba-90dd-9fd47a2a7e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:06.711489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:14:06.727424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b4f03-4979-4eba-90dd-9fd47a2a7e55 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:06.730901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8757b-3872-4582-986d-2c22f1cfbea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:06.731925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8757b-3872-4582-986d-2c22f1cfbea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:06.732228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8757b-3872-4582-986d-2c22f1cfbea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:06.732566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8757b-3872-4582-986d-2c22f1cfbea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:06.743325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8757b-3872-4582-986d-2c22f1cfbea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:06.744769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8757b-3872-4582-986d-2c22f1cfbea5 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:06.755455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 681/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:11.235331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414e82a-f6e4-4fb1-b06c-5b2bfd7280a1 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:11.235945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414e82a-f6e4-4fb1-b06c-5b2bfd7280a1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:11.235945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414e82a-f6e4-4fb1-b06c-5b2bfd7280a1 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:11.237272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 660/1341] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:11.242734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ba2e5-458d-4c00-bd0c-c039506f2a48 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:11.243249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ba2e5-458d-4c00-bd0c-c039506f2a48 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:11.243343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ba2e5-458d-4c00-bd0c-c039506f2a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:11.275023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-290ba2e5-458d-4c00-bd0c-c039506f2a48 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:11.276246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-290ba2e5-458d-4c00-bd0c-c039506f2a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:11.360960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 682/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:11.401548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414e82a-f6e4-4fb1-b06c-5b2bfd7280a1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:11.412826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414e82a-f6e4-4fb1-b06c-5b2bfd7280a1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:11.416455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa970f-4fc5-4d0a-8d5e-0f72cad4c2c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:11.417093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa970f-4fc5-4d0a-8d5e-0f72cad4c2c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:11.417379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa970f-4fc5-4d0a-8d5e-0f72cad4c2c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:11.417657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa970f-4fc5-4d0a-8d5e-0f72cad4c2c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:11.427619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa970f-4fc5-4d0a-8d5e-0f72cad4c2c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:11.429046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa970f-4fc5-4d0a-8d5e-0f72cad4c2c4 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:11.439518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 661/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:14.424324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc7b57-8857-4ea0-9e3b-a751cd204eba None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:14.424971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc7b57-8857-4ea0-9e3b-a751cd204eba None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:14.424971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc7b57-8857-4ea0-9e3b-a751cd204eba None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:14.426322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 683/1344] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:14.432030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad89fb7-9aff-41e2-90e1-43c8bdc8f30d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:14.432560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad89fb7-9aff-41e2-90e1-43c8bdc8f30d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:14.432560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad89fb7-9aff-41e2-90e1-43c8bdc8f30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:14.453489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fad89fb7-9aff-41e2-90e1-43c8bdc8f30d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:14.456665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fad89fb7-9aff-41e2-90e1-43c8bdc8f30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:14.467746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 662/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:14.508375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc7b57-8857-4ea0-9e3b-a751cd204eba None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:14.519790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc7b57-8857-4ea0-9e3b-a751cd204eba None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:14.523378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958ac21-cbea-4775-807d-dda47f9a97e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:14.524031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958ac21-cbea-4775-807d-dda47f9a97e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:14.524319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958ac21-cbea-4775-807d-dda47f9a97e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:14.524591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958ac21-cbea-4775-807d-dda47f9a97e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:14.534685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958ac21-cbea-4775-807d-dda47f9a97e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:14.536148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958ac21-cbea-4775-807d-dda47f9a97e0 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:14.550130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 684/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:19.071813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ee295-95cc-4a6a-94d0-7f3ed006051a None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:19.072373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ee295-95cc-4a6a-94d0-7f3ed006051a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:19.072373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ee295-95cc-4a6a-94d0-7f3ed006051a None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:19.073965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 663/1347] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:19.079728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddbcb1-7e1a-42f1-8ee1-42d9f56c453e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:19.080353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddbcb1-7e1a-42f1-8ee1-42d9f56c453e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:19.080353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddbcb1-7e1a-42f1-8ee1-42d9f56c453e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:19.100079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-32ddbcb1-7e1a-42f1-8ee1-42d9f56c453e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:19.101321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-32ddbcb1-7e1a-42f1-8ee1-42d9f56c453e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:19.111509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 685/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:19.149952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ee295-95cc-4a6a-94d0-7f3ed006051a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:19.162058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ee295-95cc-4a6a-94d0-7f3ed006051a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:19.165667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab1c66-c9fb-4335-8806-df53b94098b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:19.166338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1c66-c9fb-4335-8806-df53b94098b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:19.166622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1c66-c9fb-4335-8806-df53b94098b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:19.166897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1c66-c9fb-4335-8806-df53b94098b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:19.177316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab1c66-c9fb-4335-8806-df53b94098b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:19.178816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab1c66-c9fb-4335-8806-df53b94098b0 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:19.190070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 664/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:25.133728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c136ff-bee0-40ad-8b5d-ff61ce251a9d None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:25.133728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c136ff-bee0-40ad-8b5d-ff61ce251a9d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:25.133728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c136ff-bee0-40ad-8b5d-ff61ce251a9d None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:25.133728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 686/1350] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:25.141173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03004b-81af-4283-b971-44b196503c3a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:25.141173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03004b-81af-4283-b971-44b196503c3a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:25.141173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03004b-81af-4283-b971-44b196503c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:25.173010 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:14:25.328960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2d03004b-81af-4283-b971-44b196503c3a None None] MFA Rules not processed for user `51b91456a0634e0fa22fb389f0721a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:25.331029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2d03004b-81af-4283-b971-44b196503c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:25.472270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 665/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:14:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:29.094930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=57 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:14:30.096325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6312c-b84d-4024-ae23-52654da1c214 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:30.097337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6312c-b84d-4024-ae23-52654da1c214 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:30.097646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6312c-b84d-4024-ae23-52654da1c214 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:30.099409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 687/1352] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:30.105145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca60af-8793-4f24-8bd3-66f69ceda7a0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:30.105991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca60af-8793-4f24-8bd3-66f69ceda7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:30.105991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca60af-8793-4f24-8bd3-66f69ceda7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:30.127617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-35ca60af-8793-4f24-8bd3-66f69ceda7a0 None None] MFA Rules not processed for user `51b91456a0634e0fa22fb389f0721a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:30.128922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-35ca60af-8793-4f24-8bd3-66f69ceda7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:30.139318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 666/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:14:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:31.698535 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf2411-17a0-4024-a3ca-54c4b950f25e None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:31.699303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf2411-17a0-4024-a3ca-54c4b950f25e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:31.699303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf2411-17a0-4024-a3ca-54c4b950f25e None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:31.700646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 688/1354] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:31.706008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce05616-42dc-4348-9629-d91ae7c34d4f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:31.706604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce05616-42dc-4348-9629-d91ae7c34d4f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:31.706604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce05616-42dc-4348-9629-d91ae7c34d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:31.724926 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3ce05616-42dc-4348-9629-d91ae7c34d4f None None] MFA Rules not processed for user `51b91456a0634e0fa22fb389f0721a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:31.726139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3ce05616-42dc-4348-9629-d91ae7c34d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:31.736007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 667/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:14:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:35.901852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048493fd-7b42-4bab-a39d-439e2a3f00d3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:35.903150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048493fd-7b42-4bab-a39d-439e2a3f00d3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:35.903688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048493fd-7b42-4bab-a39d-439e2a3f00d3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:35.906276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 689/1356] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:35.911380 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c451b-350a-4203-b66a-75a08c1d9c31 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:35.912235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c451b-350a-4203-b66a-75a08c1d9c31 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:35.912512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c451b-350a-4203-b66a-75a08c1d9c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:35.940129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-965c451b-350a-4203-b66a-75a08c1d9c31 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:35.942716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-965c451b-350a-4203-b66a-75a08c1d9c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:35.957435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 668/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:36.024262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26288433-2f04-443a-8f88-063e396e1b04 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:36.025887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26288433-2f04-443a-8f88-063e396e1b04 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:36.027562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26288433-2f04-443a-8f88-063e396e1b04 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:36.029387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 690/1358] 10.222.0.13 () {54 vars in 961 bytes} [Mon Jun 6 14:14:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:36.040559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a6292-a08e-4f8b-9502-f804230d0e26 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:36.040847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a6292-a08e-4f8b-9502-f804230d0e26 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:36.041333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a6292-a08e-4f8b-9502-f804230d0e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:36.102139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fd7a6292-a08e-4f8b-9502-f804230d0e26 None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:36.103372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fd7a6292-a08e-4f8b-9502-f804230d0e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:36.177931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:14:36.202927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 669/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Jun 6 14:14:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:36.212061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26288433-2f04-443a-8f88-063e396e1b04 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:36.227287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26288433-2f04-443a-8f88-063e396e1b04 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:36.234192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbe3fe-7c36-484e-a1d1-156ee269aafa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:36.235071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbe3fe-7c36-484e-a1d1-156ee269aafa service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:36.235358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbe3fe-7c36-484e-a1d1-156ee269aafa service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:36.235629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbe3fe-7c36-484e-a1d1-156ee269aafa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:36.244556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbe3fe-7c36-484e-a1d1-156ee269aafa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:36.246373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbe3fe-7c36-484e-a1d1-156ee269aafa service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:36.256738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 691/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:38.937531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421d473-073f-43cc-9214-17425b4d8975 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:38.938083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421d473-073f-43cc-9214-17425b4d8975 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:38.938187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421d473-073f-43cc-9214-17425b4d8975 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:38.940312 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 670/1361] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:38.949638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d89b0-2d57-4f5d-8d57-bb6a0497d637 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:38.950254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d89b0-2d57-4f5d-8d57-bb6a0497d637 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:38.950672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d89b0-2d57-4f5d-8d57-bb6a0497d637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:38.973779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3b7d89b0-2d57-4f5d-8d57-bb6a0497d637 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:38.975592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3b7d89b0-2d57-4f5d-8d57-bb6a0497d637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:38.985897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 692/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:39.037655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421d473-073f-43cc-9214-17425b4d8975 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:39.054834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421d473-073f-43cc-9214-17425b4d8975 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:39.062510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927d06f-d83f-4606-bcd3-620976434f5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:39.063285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927d06f-d83f-4606-bcd3-620976434f5f service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:39.063664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927d06f-d83f-4606-bcd3-620976434f5f service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:39.064016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927d06f-d83f-4606-bcd3-620976434f5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:39.074635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927d06f-d83f-4606-bcd3-620976434f5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:39.076761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927d06f-d83f-4606-bcd3-620976434f5f service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:39.092702 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 671/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:41.866298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be5d97-6889-4a43-96f9-0b2ac4a92ccb None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:41.867139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be5d97-6889-4a43-96f9-0b2ac4a92ccb None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:41.867139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be5d97-6889-4a43-96f9-0b2ac4a92ccb None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:41.868143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 693/1364] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:41.874687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd630fb8-beea-4e55-8922-e209ad84a16e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:41.875416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd630fb8-beea-4e55-8922-e209ad84a16e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:41.875416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd630fb8-beea-4e55-8922-e209ad84a16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:41.899055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-dd630fb8-beea-4e55-8922-e209ad84a16e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:41.900308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dd630fb8-beea-4e55-8922-e209ad84a16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:41.909893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 672/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:41.958322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc6384-987d-402a-b046-a27961da5c60 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:41.958567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc6384-987d-402a-b046-a27961da5c60 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:41.959024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc6384-987d-402a-b046-a27961da5c60 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:41.960589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 694/1366] 10.222.0.13 () {54 vars in 961 bytes} [Mon Jun 6 14:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:41.969417 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfad2f-be40-4b8f-bb10-da44d72b6880 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:41.969698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfad2f-be40-4b8f-bb10-da44d72b6880 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:41.970054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfad2f-be40-4b8f-bb10-da44d72b6880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:41.993993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c4dfad2f-be40-4b8f-bb10-da44d72b6880 None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:41.995295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c4dfad2f-be40-4b8f-bb10-da44d72b6880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:42.004968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 673/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Jun 6 14:14:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:42.013787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc6384-987d-402a-b046-a27961da5c60 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:42.032172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc6384-987d-402a-b046-a27961da5c60 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:42.038327 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6605a-59b6-481c-9261-92d96bea34d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:42.038989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6605a-59b6-481c-9261-92d96bea34d6 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:42.039281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6605a-59b6-481c-9261-92d96bea34d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:42.039569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6605a-59b6-481c-9261-92d96bea34d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:42.049972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6605a-59b6-481c-9261-92d96bea34d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:42.051626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6605a-59b6-481c-9261-92d96bea34d6 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:42.062847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 695/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:44.553533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3499c9-e8d3-49a9-bc9d-8a28c073a3b8 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:44.554669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3499c9-e8d3-49a9-bc9d-8a28c073a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:44.555189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3499c9-e8d3-49a9-bc9d-8a28c073a3b8 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:44.557583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 674/1369] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:14:44.563965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555ec23-ba70-43ee-b770-8640c5e1570d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:44.564809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555ec23-ba70-43ee-b770-8640c5e1570d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:44.564809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555ec23-ba70-43ee-b770-8640c5e1570d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:44.584064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8555ec23-ba70-43ee-b770-8640c5e1570d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:44.585319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8555ec23-ba70-43ee-b770-8640c5e1570d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:44.594813 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 696/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:14:44.645551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3499c9-e8d3-49a9-bc9d-8a28c073a3b8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:44.662012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3499c9-e8d3-49a9-bc9d-8a28c073a3b8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:44.668626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6ceba-99f5-447d-b778-2480b7d43338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:44.669284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6ceba-99f5-447d-b778-2480b7d43338 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:44.669785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6ceba-99f5-447d-b778-2480b7d43338 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:44.670232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6ceba-99f5-447d-b778-2480b7d43338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:44.682713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6ceba-99f5-447d-b778-2480b7d43338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:44.685978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6ceba-99f5-447d-b778-2480b7d43338 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:44.704542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 675/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:45.412003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58f4fd-782f-48c7-8a86-a250067335c4 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:45.412597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58f4fd-782f-48c7-8a86-a250067335c4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:45.413041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58f4fd-782f-48c7-8a86-a250067335c4 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:45.415295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 697/1372] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:45.421798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74bc2c-fe46-4d2c-9f82-b421088a7d07 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:45.422322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74bc2c-fe46-4d2c-9f82-b421088a7d07 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:45.422795 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74bc2c-fe46-4d2c-9f82-b421088a7d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:45.447123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0d74bc2c-fe46-4d2c-9f82-b421088a7d07 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:45.449938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0d74bc2c-fe46-4d2c-9f82-b421088a7d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:45.460262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 676/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:45.864963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea589c5-194b-4e1d-bd7a-c5f774089393 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:45.865519 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea589c5-194b-4e1d-bd7a-c5f774089393 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:45.865519 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea589c5-194b-4e1d-bd7a-c5f774089393 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:45.866908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 698/1374] 10.222.0.13 () {54 vars in 963 bytes} [Mon Jun 6 14:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:45.875011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2bf9-1d41-45ae-a106-1e4fa9555e5c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:45.875572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2bf9-1d41-45ae-a106-1e4fa9555e5c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:45.875942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb2bf9-1d41-45ae-a106-1e4fa9555e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:45.969596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9dbb2bf9-1d41-45ae-a106-1e4fa9555e5c None None] MFA Rules not processed for user `720c995318a24e7b859fa82febd8178b`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:45.971207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9dbb2bf9-1d41-45ae-a106-1e4fa9555e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:46.060503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 677/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Mon Jun 6 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:46.075128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea589c5-194b-4e1d-bd7a-c5f774089393 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:46.093610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea589c5-194b-4e1d-bd7a-c5f774089393 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:46.099620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36eb8c-3ac9-4626-8474-9d24a3494715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:46.100633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36eb8c-3ac9-4626-8474-9d24a3494715 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:46.101041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36eb8c-3ac9-4626-8474-9d24a3494715 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:46.101427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36eb8c-3ac9-4626-8474-9d24a3494715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:46.118021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36eb8c-3ac9-4626-8474-9d24a3494715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:46.120322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36eb8c-3ac9-4626-8474-9d24a3494715 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:46.141168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 699/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:47.265842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9cb7-9e65-4deb-9c57-8f009c8bb113 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:47.266641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9cb7-9e65-4deb-9c57-8f009c8bb113 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:47.266641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9cb7-9e65-4deb-9c57-8f009c8bb113 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:47.267802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 678/1377] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:14:47.277995 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ccc08-c923-456c-836b-d47c316dd430 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:47.278752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ccc08-c923-456c-836b-d47c316dd430 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:47.278897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ccc08-c923-456c-836b-d47c316dd430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:47.311536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-094ccc08-c923-456c-836b-d47c316dd430 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:47.312882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-094ccc08-c923-456c-836b-d47c316dd430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:47.325068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 700/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:14:47.401863 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f9cb7-9e65-4deb-9c57-8f009c8bb113 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:47.419975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f9cb7-9e65-4deb-9c57-8f009c8bb113 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:47.426303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212caec-b187-446e-b09e-6c6472a71cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:47.427038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212caec-b187-446e-b09e-6c6472a71cb6 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:47.427397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212caec-b187-446e-b09e-6c6472a71cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:47.427823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212caec-b187-446e-b09e-6c6472a71cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:47.438436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212caec-b187-446e-b09e-6c6472a71cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:47.440114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212caec-b187-446e-b09e-6c6472a71cb6 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:47.454794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 679/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:49.996338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db66e57-9437-4e02-8353-00b5d824f782 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:49.997098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db66e57-9437-4e02-8353-00b5d824f782 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:49.997098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db66e57-9437-4e02-8353-00b5d824f782 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:49.998133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 701/1380] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:14:50.006958 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5af7f-fcbe-45e7-b932-a51bead62c22 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:50.007553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5af7f-fcbe-45e7-b932-a51bead62c22 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:50.007749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5af7f-fcbe-45e7-b932-a51bead62c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:50.037418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-61e5af7f-fcbe-45e7-b932-a51bead62c22 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:50.039537 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-61e5af7f-fcbe-45e7-b932-a51bead62c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:50.055485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 680/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:50.127376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db66e57-9437-4e02-8353-00b5d824f782 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:50.152237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db66e57-9437-4e02-8353-00b5d824f782 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:50.163541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991f119-2ad9-45a2-aab8-229e68deef62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:50.166428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991f119-2ad9-45a2-aab8-229e68deef62 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:50.167413 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991f119-2ad9-45a2-aab8-229e68deef62 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:50.168318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991f119-2ad9-45a2-aab8-229e68deef62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:50.178104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991f119-2ad9-45a2-aab8-229e68deef62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:50.179733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991f119-2ad9-45a2-aab8-229e68deef62 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:50.191516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 702/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:14:53.306551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5031bc-77c4-4623-bd59-2ed1c8aa7da6 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:53.307333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5031bc-77c4-4623-bd59-2ed1c8aa7da6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:53.307333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5031bc-77c4-4623-bd59-2ed1c8aa7da6 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:53.308606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 681/1383] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:53.315190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cadcd5-0a9a-41b8-81db-7cacc9c3d8aa None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:53.316122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cadcd5-0a9a-41b8-81db-7cacc9c3d8aa None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:53.316122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cadcd5-0a9a-41b8-81db-7cacc9c3d8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:53.340483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-00cadcd5-0a9a-41b8-81db-7cacc9c3d8aa None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:53.346336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-00cadcd5-0a9a-41b8-81db-7cacc9c3d8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:53.365855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 703/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:14:53.409375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5031bc-77c4-4623-bd59-2ed1c8aa7da6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:53.425723 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5031bc-77c4-4623-bd59-2ed1c8aa7da6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:53.431883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44071cf-3766-43a0-b90c-db365c0ab1ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:53.432801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44071cf-3766-43a0-b90c-db365c0ab1ef service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:53.433247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44071cf-3766-43a0-b90c-db365c0ab1ef service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:53.433624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44071cf-3766-43a0-b90c-db365c0ab1ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:53.443678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44071cf-3766-43a0-b90c-db365c0ab1ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:53.445500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44071cf-3766-43a0-b90c-db365c0ab1ef service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:53.458005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 682/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:55.909290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac708f4f-3238-4327-abb3-884b482e28f3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:55.909758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac708f4f-3238-4327-abb3-884b482e28f3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:55.909758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac708f4f-3238-4327-abb3-884b482e28f3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:55.911217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 704/1386] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:14:55.917502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489197a1-c79e-4341-9366-545e839e9615 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:55.917502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489197a1-c79e-4341-9366-545e839e9615 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:55.917502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489197a1-c79e-4341-9366-545e839e9615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:55.938079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-489197a1-c79e-4341-9366-545e839e9615 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:55.939288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-489197a1-c79e-4341-9366-545e839e9615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:55.949867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 683/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:14:55.994256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac708f4f-3238-4327-abb3-884b482e28f3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:56.015566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac708f4f-3238-4327-abb3-884b482e28f3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:56.022424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36565117-7e82-4057-873d-e33791f3f75f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:56.023177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36565117-7e82-4057-873d-e33791f3f75f service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:56.023540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36565117-7e82-4057-873d-e33791f3f75f service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:56.023730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36565117-7e82-4057-873d-e33791f3f75f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:56.035600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36565117-7e82-4057-873d-e33791f3f75f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:56.037374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36565117-7e82-4057-873d-e33791f3f75f service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:56.049068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 705/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:14:58.649527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3860ee-6b40-4f72-a669-6608987a45e1 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:58.650744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3860ee-6b40-4f72-a669-6608987a45e1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:58.650744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3860ee-6b40-4f72-a669-6608987a45e1 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:58.652509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 684/1389] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:14:58.658062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a21e-4c39-4f70-9fe7-7bbe8fdc865b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:58.658523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a21e-4c39-4f70-9fe7-7bbe8fdc865b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:58.658649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a21e-4c39-4f70-9fe7-7bbe8fdc865b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:58.677542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7fa9a21e-4c39-4f70-9fe7-7bbe8fdc865b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:14:58.678846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7fa9a21e-4c39-4f70-9fe7-7bbe8fdc865b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:14:58.688782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 706/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:14:58.747929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3860ee-6b40-4f72-a669-6608987a45e1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:14:58.765793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3860ee-6b40-4f72-a669-6608987a45e1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:14:58.772504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637b737-c471-4ad7-9ebd-b78fd0543c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:14:58.773384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637b737-c471-4ad7-9ebd-b78fd0543c53 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:14:58.773670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637b737-c471-4ad7-9ebd-b78fd0543c53 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:14:58.774003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637b737-c471-4ad7-9ebd-b78fd0543c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:14:58.784239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637b737-c471-4ad7-9ebd-b78fd0543c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:14:58.785917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637b737-c471-4ad7-9ebd-b78fd0543c53 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:14:58.800686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 685/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:01.294163 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406b425-5bc5-402a-ba7b-27986dcc6294 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:01.294851 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406b425-5bc5-402a-ba7b-27986dcc6294 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:01.294980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406b425-5bc5-402a-ba7b-27986dcc6294 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:01.297110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 707/1392] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:15:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:15:01.303983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31662591-b64a-49ca-8e01-f7eb126fbb47 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:01.304731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31662591-b64a-49ca-8e01-f7eb126fbb47 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:01.304920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31662591-b64a-49ca-8e01-f7eb126fbb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:01.331590 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-31662591-b64a-49ca-8e01-f7eb126fbb47 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:01.333509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-31662591-b64a-49ca-8e01-f7eb126fbb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:01.346196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 686/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:15:01.416738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406b425-5bc5-402a-ba7b-27986dcc6294 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:01.426755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=46 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:15:01.439266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406b425-5bc5-402a-ba7b-27986dcc6294 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:01.453339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e30e2-e2e8-4e9a-bab3-eebc632ad764 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:01.455159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e30e2-e2e8-4e9a-bab3-eebc632ad764 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:01.456087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e30e2-e2e8-4e9a-bab3-eebc632ad764 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:01.456412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e30e2-e2e8-4e9a-bab3-eebc632ad764 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:01.468489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e30e2-e2e8-4e9a-bab3-eebc632ad764 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:01.471311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e30e2-e2e8-4e9a-bab3-eebc632ad764 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:01.491599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 708/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:03.956775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa4982-a1a1-4a60-91a2-4ade49f2e0a9 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:03.957396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa4982-a1a1-4a60-91a2-4ade49f2e0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:03.957832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa4982-a1a1-4a60-91a2-4ade49f2e0a9 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:03.960380 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 687/1395] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:15:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:03.966729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b37677-12d2-40a6-84e7-8ee78be0bdc9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:03.967319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b37677-12d2-40a6-84e7-8ee78be0bdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:03.967319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b37677-12d2-40a6-84e7-8ee78be0bdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:03.991310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-84b37677-12d2-40a6-84e7-8ee78be0bdc9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:03.994169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-84b37677-12d2-40a6-84e7-8ee78be0bdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:04.008505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 709/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:04.034188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac3a55-23ee-455a-b432-666cbce5e949 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:04.034650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac3a55-23ee-455a-b432-666cbce5e949 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:04.035120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac3a55-23ee-455a-b432-666cbce5e949 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:04.036978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 688/1397] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:04.044740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a3957-b623-436c-975d-b046ed3d13e4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:04.045003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a3957-b623-436c-975d-b046ed3d13e4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:04.045281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a3957-b623-436c-975d-b046ed3d13e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:04.068577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ba38-4c92-46b5-b1ea-d4bb86595701 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:04.068965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ba38-4c92-46b5-b1ea-d4bb86595701 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:04.069450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ba38-4c92-46b5-b1ea-d4bb86595701 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:04.071170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 689/1398] 10.222.0.13 () {54 vars in 962 bytes} [Mon Jun 6 14:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:15:04.071837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-243a3957-b623-436c-975d-b046ed3d13e4 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:04.073208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-243a3957-b623-436c-975d-b046ed3d13e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:04.077690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3af4ec-8c8e-4434-a2c9-9c4cc75bc8c8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:04.078142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3af4ec-8c8e-4434-a2c9-9c4cc75bc8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:04.078520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3af4ec-8c8e-4434-a2c9-9c4cc75bc8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:04.081596 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 710/1399] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:04.112432 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:15:04.131460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a3957-b623-436c-975d-b046ed3d13e4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:04.140528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6e3af4ec-8c8e-4434-a2c9-9c4cc75bc8c8 None None] MFA Rules not processed for user `f41a357957bc47739b7bd6f21d159d74`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:04.142123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6e3af4ec-8c8e-4434-a2c9-9c4cc75bc8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:04.144254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a3957-b623-436c-975d-b046ed3d13e4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:04.150482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9163783-5b60-409a-a0f9-aae1a7c582de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:04.151135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9163783-5b60-409a-a0f9-aae1a7c582de service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:04.151445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9163783-5b60-409a-a0f9-aae1a7c582de service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:04.151696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9163783-5b60-409a-a0f9-aae1a7c582de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:04.160679 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9163783-5b60-409a-a0f9-aae1a7c582de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:04.163696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9163783-5b60-409a-a0f9-aae1a7c582de service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:04.173977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 711/1400] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:15:04.228892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 690/1401] 10.222.0.13 () {58 vars in 1059 bytes} [Mon Jun 6 14:15:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:15:04.238254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9163783-5b60-409a-a0f9-aae1a7c582de service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:04.249901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9163783-5b60-409a-a0f9-aae1a7c582de service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:04.254045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575f600-db12-41aa-a82d-7bb70639fea2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41a357957bc47739b7bd6f21d159d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:04.255075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575f600-db12-41aa-a82d-7bb70639fea2 service glance] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:04.255552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575f600-db12-41aa-a82d-7bb70639fea2 service glance] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:04.256093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575f600-db12-41aa-a82d-7bb70639fea2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:04.272825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575f600-db12-41aa-a82d-7bb70639fea2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:04.275377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575f600-db12-41aa-a82d-7bb70639fea2 service glance] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:04.291899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 712/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:04.888124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb9e00-87af-4fa4-972e-6d11b6f3e4c3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:04.888453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb9e00-87af-4fa4-972e-6d11b6f3e4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:04.888762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb9e00-87af-4fa4-972e-6d11b6f3e4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:04.952079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-86eb9e00-87af-4fa4-972e-6d11b6f3e4c3 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:04.953968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-86eb9e00-87af-4fa4-972e-6d11b6f3e4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:05.033373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 691/1403] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:15:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:05.083021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a9a38-6b11-4f39-ab5c-5ca13f19af91 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:05.083575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a9a38-6b11-4f39-ab5c-5ca13f19af91 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:05.083693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a9a38-6b11-4f39-ab5c-5ca13f19af91 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:05.085031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 713/1404] 10.222.0.13 () {54 vars in 942 bytes} [Mon Jun 6 14:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:05.092299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb2685-4214-403c-a40e-d619a42ce1e7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:05.092654 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb2685-4214-403c-a40e-d619a42ce1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:05.093024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb2685-4214-403c-a40e-d619a42ce1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:05.151866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8ecb2685-4214-403c-a40e-d619a42ce1e7 None None] MFA Rules not processed for user `3571f3d3a0de4e9abe55ed84426143a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:05.153861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8ecb2685-4214-403c-a40e-d619a42ce1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:05.228099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:15:05.267092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 692/1405] 10.222.0.13 () {58 vars in 1039 bytes} [Mon Jun 6 14:15:05 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:05.277752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a9a38-6b11-4f39-ab5c-5ca13f19af91 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:05.291928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a9a38-6b11-4f39-ab5c-5ca13f19af91 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:05.296964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4933e3-44e8-4704-9e6e-a03eb2e1612d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:05.297620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4933e3-44e8-4704-9e6e-a03eb2e1612d service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:05.297935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4933e3-44e8-4704-9e6e-a03eb2e1612d service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:05.298215 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4933e3-44e8-4704-9e6e-a03eb2e1612d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:05.306943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4933e3-44e8-4704-9e6e-a03eb2e1612d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:05.308538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4933e3-44e8-4704-9e6e-a03eb2e1612d service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:05.316866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 714/1406] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:15:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:15:06.746573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045798b-dd12-4bd5-be0a-8e9a473ef546 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:06.747908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045798b-dd12-4bd5-be0a-8e9a473ef546 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:06.748754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045798b-dd12-4bd5-be0a-8e9a473ef546 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:06.754065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 693/1407] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:15:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:06.767013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c5120-60da-49da-b968-1c342dfe6e2b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:06.768313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c5120-60da-49da-b968-1c342dfe6e2b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:06.768926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c5120-60da-49da-b968-1c342dfe6e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:06.803580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-240c5120-60da-49da-b968-1c342dfe6e2b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:06.806814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-240c5120-60da-49da-b968-1c342dfe6e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:06.826208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 715/1408] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:06.903851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045798b-dd12-4bd5-be0a-8e9a473ef546 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:06.926545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045798b-dd12-4bd5-be0a-8e9a473ef546 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:06.933660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f7efa-a977-4566-b0b1-88760585a9d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:06.936560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7efa-a977-4566-b0b1-88760585a9d7 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:06.937179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7efa-a977-4566-b0b1-88760585a9d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:06.937789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7efa-a977-4566-b0b1-88760585a9d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:06.949464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f7efa-a977-4566-b0b1-88760585a9d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:06.951117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f7efa-a977-4566-b0b1-88760585a9d7 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:06.962919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 694/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:15:16.794787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=32 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:15:16.957244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=19 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:15:30.121207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e716ef-6bfa-423a-8db3-5c96c37ce1a2 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:30.121881 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e716ef-6bfa-423a-8db3-5c96c37ce1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:30.121881 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e716ef-6bfa-423a-8db3-5c96c37ce1a2 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:30.123185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 716/1410] 10.222.0.13 () {54 vars in 918 bytes} [Mon Jun 6 14:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:30.128680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fd3e9-a3ea-4037-8338-c4dbcda4f0df None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:30.129207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fd3e9-a3ea-4037-8338-c4dbcda4f0df None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:30.129207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fd3e9-a3ea-4037-8338-c4dbcda4f0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:30.150300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6c9fd3e9-a3ea-4037-8338-c4dbcda4f0df None None] MFA Rules not processed for user `51b91456a0634e0fa22fb389f0721a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:30.151515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6c9fd3e9-a3ea-4037-8338-c4dbcda4f0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:30.160169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 695/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Mon Jun 6 14:15:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:15:30.175047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef4696-34ae-46f9-8c4a-e39d486f29d4 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:30.175335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef4696-34ae-46f9-8c4a-e39d486f29d4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:30.175611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef4696-34ae-46f9-8c4a-e39d486f29d4 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:30.177035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 717/1412] 10.222.0.13 () {54 vars in 918 bytes} [Mon Jun 6 14:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:30.183013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf2d19-5de1-4ee6-a811-5c38b9a9127f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:30.183302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf2d19-5de1-4ee6-a811-5c38b9a9127f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:30.183606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf2d19-5de1-4ee6-a811-5c38b9a9127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:30.202698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d2bf2d19-5de1-4ee6-a811-5c38b9a9127f None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:30.203908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d2bf2d19-5de1-4ee6-a811-5c38b9a9127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:30.213863 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 696/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Mon Jun 6 14:15:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:33.046531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73191a19-e55f-4753-b8da-d6009602669d None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:33.047129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73191a19-e55f-4753-b8da-d6009602669d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:33.047129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73191a19-e55f-4753-b8da-d6009602669d None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:33.048456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:33.054031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5db13d-4c88-4a5a-bcf4-340468305eaf None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:33.055136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5db13d-4c88-4a5a-bcf4-340468305eaf None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:33.055315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5db13d-4c88-4a5a-bcf4-340468305eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:33.075820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7d5db13d-4c88-4a5a-bcf4-340468305eaf None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:33.077012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7d5db13d-4c88-4a5a-bcf4-340468305eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:33.084912 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:33.155738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73191a19-e55f-4753-b8da-d6009602669d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:33.172944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73191a19-e55f-4753-b8da-d6009602669d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:33.181378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e65edc-4685-4ad8-b5b7-8ac58e0899fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:33.182082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65edc-4685-4ad8-b5b7-8ac58e0899fa service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:33.182371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65edc-4685-4ad8-b5b7-8ac58e0899fa service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:33.182657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65edc-4685-4ad8-b5b7-8ac58e0899fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:33.191693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e65edc-4685-4ad8-b5b7-8ac58e0899fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:33.193301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e65edc-4685-4ad8-b5b7-8ac58e0899fa service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:33.203112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 719/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:34.219884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81573831-0547-470a-9d65-ef3ffa11e2a7 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:34.220452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81573831-0547-470a-9d65-ef3ffa11e2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:34.220452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81573831-0547-470a-9d65-ef3ffa11e2a7 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:34.221783 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:15:34.228471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a285a-58ea-4c59-a2fb-d3c94849ea5e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:34.228972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a285a-58ea-4c59-a2fb-d3c94849ea5e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:34.229083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a285a-58ea-4c59-a2fb-d3c94849ea5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:34.248032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5e8a285a-58ea-4c59-a2fb-d3c94849ea5e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:34.249247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5e8a285a-58ea-4c59-a2fb-d3c94849ea5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:34.257065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:34.435261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81573831-0547-470a-9d65-ef3ffa11e2a7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:34.453148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81573831-0547-470a-9d65-ef3ffa11e2a7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:34.459602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017d305-a369-4391-b80d-d84cb84ec3fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:34.460262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d305-a369-4391-b80d-d84cb84ec3fe service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:34.460596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d305-a369-4391-b80d-d84cb84ec3fe service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:34.460877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d305-a369-4391-b80d-d84cb84ec3fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:34.471441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017d305-a369-4391-b80d-d84cb84ec3fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:34.473190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017d305-a369-4391-b80d-d84cb84ec3fe service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:34.485517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 699/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:35.573211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c15d5-16fe-4e8a-8268-3821852d6223 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:35.573867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c15d5-16fe-4e8a-8268-3821852d6223 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:35.573867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c15d5-16fe-4e8a-8268-3821852d6223 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:35.575222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:35.581013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7261c8-e2e1-4970-a382-de87e5f443d0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:35.581637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7261c8-e2e1-4970-a382-de87e5f443d0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:35.581637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7261c8-e2e1-4970-a382-de87e5f443d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:35.604876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5c7261c8-e2e1-4970-a382-de87e5f443d0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:35.609870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5c7261c8-e2e1-4970-a382-de87e5f443d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:35.620751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:35.827187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c15d5-16fe-4e8a-8268-3821852d6223 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:35.851131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c15d5-16fe-4e8a-8268-3821852d6223 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:35.858731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607ad8a-60cc-4be5-bffb-32d11ee8096c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:35.859381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607ad8a-60cc-4be5-bffb-32d11ee8096c service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:35.859663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607ad8a-60cc-4be5-bffb-32d11ee8096c service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:35.859938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607ad8a-60cc-4be5-bffb-32d11ee8096c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:35.875660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607ad8a-60cc-4be5-bffb-32d11ee8096c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:35.877284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607ad8a-60cc-4be5-bffb-32d11ee8096c service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:35.888582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 722/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:36.819224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f8842-8ba4-4ef5-b21b-baf90010594a None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:36.820393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f8842-8ba4-4ef5-b21b-baf90010594a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:36.820393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f8842-8ba4-4ef5-b21b-baf90010594a None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:36.821628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:36.828797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ddc34-b63a-410a-b133-cfa067ee60e1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:36.829264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ddc34-b63a-410a-b133-cfa067ee60e1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:36.829361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ddc34-b63a-410a-b133-cfa067ee60e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:36.849374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bc0ddc34-b63a-410a-b133-cfa067ee60e1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:36.850622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bc0ddc34-b63a-410a-b133-cfa067ee60e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:36.858981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:37.009729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f8842-8ba4-4ef5-b21b-baf90010594a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:37.025361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f8842-8ba4-4ef5-b21b-baf90010594a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:37.030751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53145a-f72a-4a48-8cd6-b59df15f303c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:37.031394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53145a-f72a-4a48-8cd6-b59df15f303c service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:37.031684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53145a-f72a-4a48-8cd6-b59df15f303c service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:37.031959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53145a-f72a-4a48-8cd6-b59df15f303c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:37.040631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53145a-f72a-4a48-8cd6-b59df15f303c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:37.042254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53145a-f72a-4a48-8cd6-b59df15f303c service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:37.052278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 702/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:38.023746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6698cd-d1f7-401d-861f-1edec15f93d3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:38.024297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6698cd-d1f7-401d-861f-1edec15f93d3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:38.024297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6698cd-d1f7-401d-861f-1edec15f93d3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:38.025664 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:15:38.030882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670278ab-48c1-4007-b163-4b4951c6cf1e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:38.031467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670278ab-48c1-4007-b163-4b4951c6cf1e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:38.031467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670278ab-48c1-4007-b163-4b4951c6cf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:38.051919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-670278ab-48c1-4007-b163-4b4951c6cf1e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:38.053129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-670278ab-48c1-4007-b163-4b4951c6cf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:38.061583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:38.218214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6698cd-d1f7-401d-861f-1edec15f93d3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:38.234742 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6698cd-d1f7-401d-861f-1edec15f93d3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:38.240122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bf822-9fed-4206-84ab-e821918c8453 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:38.240765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bf822-9fed-4206-84ab-e821918c8453 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:38.241050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bf822-9fed-4206-84ab-e821918c8453 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:38.241323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bf822-9fed-4206-84ab-e821918c8453 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:38.250715 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bf822-9fed-4206-84ab-e821918c8453 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:38.252322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bf822-9fed-4206-84ab-e821918c8453 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:38.262457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 725/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:15:39.150662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f294d1-3ee3-4409-a32d-6294bd1cb7b6 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:39.151225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f294d1-3ee3-4409-a32d-6294bd1cb7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:39.151225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f294d1-3ee3-4409-a32d-6294bd1cb7b6 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:39.152644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:39.158622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0b260-cba5-4f25-be2d-aa30d83090c1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:39.159209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0b260-cba5-4f25-be2d-aa30d83090c1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:39.159209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0b260-cba5-4f25-be2d-aa30d83090c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:39.182029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b5e0b260-cba5-4f25-be2d-aa30d83090c1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:39.183658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b5e0b260-cba5-4f25-be2d-aa30d83090c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:39.194022 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:15:39.362703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f294d1-3ee3-4409-a32d-6294bd1cb7b6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:39.378196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f294d1-3ee3-4409-a32d-6294bd1cb7b6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:39.384394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911b1fc-6de5-4bb1-a899-1d0e6a9409af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:39.385039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911b1fc-6de5-4bb1-a899-1d0e6a9409af service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:39.385322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911b1fc-6de5-4bb1-a899-1d0e6a9409af service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:39.385604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911b1fc-6de5-4bb1-a899-1d0e6a9409af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:39.394540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911b1fc-6de5-4bb1-a899-1d0e6a9409af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:39.396187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911b1fc-6de5-4bb1-a899-1d0e6a9409af service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:39.406757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 705/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:40.276759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4cb4d-dce9-40c6-8d1f-8e57d18c4e9d None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:40.277662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4cb4d-dce9-40c6-8d1f-8e57d18c4e9d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:40.277662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4cb4d-dce9-40c6-8d1f-8e57d18c4e9d None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:40.278721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:40.283705 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b581b8-18dd-4b5c-a78a-e5d9d4ec4187 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:40.284191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b581b8-18dd-4b5c-a78a-e5d9d4ec4187 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:40.284327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b581b8-18dd-4b5c-a78a-e5d9d4ec4187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:40.307296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f5b581b8-18dd-4b5c-a78a-e5d9d4ec4187 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:40.308500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f5b581b8-18dd-4b5c-a78a-e5d9d4ec4187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:40.316445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:40.470132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4cb4d-dce9-40c6-8d1f-8e57d18c4e9d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:40.488912 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4cb4d-dce9-40c6-8d1f-8e57d18c4e9d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:40.495337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659827d1-bd4d-410d-a2c8-a313dec9ec6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:40.496161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659827d1-bd4d-410d-a2c8-a313dec9ec6d service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:40.496487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659827d1-bd4d-410d-a2c8-a313dec9ec6d service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:40.496804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659827d1-bd4d-410d-a2c8-a313dec9ec6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:40.508192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659827d1-bd4d-410d-a2c8-a313dec9ec6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:40.513528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659827d1-bd4d-410d-a2c8-a313dec9ec6d service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:40.521846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 728/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:15:41.435562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43252-1b57-464a-b569-6b1e9c190990 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:41.436139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43252-1b57-464a-b569-6b1e9c190990 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:41.436139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43252-1b57-464a-b569-6b1e9c190990 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:41.437539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:15:41.442699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b87d8-4c05-4dcb-9276-97eb2547e82c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:41.443363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b87d8-4c05-4dcb-9276-97eb2547e82c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:41.443609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b87d8-4c05-4dcb-9276-97eb2547e82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:41.464842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-902b87d8-4c05-4dcb-9276-97eb2547e82c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:41.466079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-902b87d8-4c05-4dcb-9276-97eb2547e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:41.473873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:41.634623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43252-1b57-464a-b569-6b1e9c190990 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:41.649947 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43252-1b57-464a-b569-6b1e9c190990 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:41.655475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06268db6-32ff-426b-8abe-4ae78ce03998 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:41.656120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268db6-32ff-426b-8abe-4ae78ce03998 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:41.656399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268db6-32ff-426b-8abe-4ae78ce03998 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:41.656673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268db6-32ff-426b-8abe-4ae78ce03998 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:41.666233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06268db6-32ff-426b-8abe-4ae78ce03998 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:41.667833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06268db6-32ff-426b-8abe-4ae78ce03998 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:41.677569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 708/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:42.571527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631adf0-8355-4772-a683-5a6d3458d5be None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:42.572072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631adf0-8355-4772-a683-5a6d3458d5be None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:42.572072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631adf0-8355-4772-a683-5a6d3458d5be None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:42.573460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:42.578931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095579a-f1c8-46db-b798-12ec8feb46d9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:42.579447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095579a-f1c8-46db-b798-12ec8feb46d9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:42.579554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095579a-f1c8-46db-b798-12ec8feb46d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:42.597940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6095579a-f1c8-46db-b798-12ec8feb46d9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:42.599129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6095579a-f1c8-46db-b798-12ec8feb46d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:42.607549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:42.766994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631adf0-8355-4772-a683-5a6d3458d5be None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:42.782038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631adf0-8355-4772-a683-5a6d3458d5be None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:42.787462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb873d-8ecc-4d16-9014-f2ad208b3920 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:42.788643 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb873d-8ecc-4d16-9014-f2ad208b3920 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:42.789255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb873d-8ecc-4d16-9014-f2ad208b3920 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:42.789815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb873d-8ecc-4d16-9014-f2ad208b3920 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:42.799197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb873d-8ecc-4d16-9014-f2ad208b3920 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:42.800808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb873d-8ecc-4d16-9014-f2ad208b3920 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:42.812171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 731/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:45.027908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee670677-ffb7-49fe-9e18-49efd3adfe3e None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:45.028515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee670677-ffb7-49fe-9e18-49efd3adfe3e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:45.028515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee670677-ffb7-49fe-9e18-49efd3adfe3e None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:45.029856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:45.035402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eda49-bd79-4187-a0d4-9cf9086ddc71 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:45.035928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eda49-bd79-4187-a0d4-9cf9086ddc71 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:45.036025 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eda49-bd79-4187-a0d4-9cf9086ddc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:45.055051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-249eda49-bd79-4187-a0d4-9cf9086ddc71 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:45.056257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-249eda49-bd79-4187-a0d4-9cf9086ddc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:45.064584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:45.106764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee670677-ffb7-49fe-9e18-49efd3adfe3e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:45.123767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee670677-ffb7-49fe-9e18-49efd3adfe3e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:45.130099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a603cd-6f06-4a03-83d3-f254012d01a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:45.131423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a603cd-6f06-4a03-83d3-f254012d01a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:45.132736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a603cd-6f06-4a03-83d3-f254012d01a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:45.133262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a603cd-6f06-4a03-83d3-f254012d01a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:45.151225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a603cd-6f06-4a03-83d3-f254012d01a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:45.153955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a603cd-6f06-4a03-83d3-f254012d01a2 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:45.173665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 711/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:47.689889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd992e-072a-45f2-915f-f68559bd79db None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:47.690488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd992e-072a-45f2-915f-f68559bd79db None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:47.690488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd992e-072a-45f2-915f-f68559bd79db None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:47.691876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:47.698766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829692ce-c458-40aa-b5b1-9636d8935176 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:47.699311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829692ce-c458-40aa-b5b1-9636d8935176 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:47.699311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829692ce-c458-40aa-b5b1-9636d8935176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:47.718408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-829692ce-c458-40aa-b5b1-9636d8935176 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:47.719636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-829692ce-c458-40aa-b5b1-9636d8935176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:47.728188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:47.769025 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd992e-072a-45f2-915f-f68559bd79db None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:47.781122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd992e-072a-45f2-915f-f68559bd79db None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:47.784963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882c22c-6ea5-418a-9bac-4025d1d348ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:47.785633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882c22c-6ea5-418a-9bac-4025d1d348ce service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:47.785966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882c22c-6ea5-418a-9bac-4025d1d348ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:47.786394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882c22c-6ea5-418a-9bac-4025d1d348ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:47.797482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882c22c-6ea5-418a-9bac-4025d1d348ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:47.798999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882c22c-6ea5-418a-9bac-4025d1d348ce service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:47.810334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 734/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:50.436163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65535ec-0711-41d8-b7c4-a75ffa2067a3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:50.436782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65535ec-0711-41d8-b7c4-a75ffa2067a3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:50.436782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65535ec-0711-41d8-b7c4-a75ffa2067a3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:50.438289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:50.444371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998215da-33fc-4d36-8970-e423b67f8787 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:50.444896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998215da-33fc-4d36-8970-e423b67f8787 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:50.445084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998215da-33fc-4d36-8970-e423b67f8787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:50.463693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-998215da-33fc-4d36-8970-e423b67f8787 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:50.464902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-998215da-33fc-4d36-8970-e423b67f8787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:50.472955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:50.512495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65535ec-0711-41d8-b7c4-a75ffa2067a3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:50.514351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a65535ec-0711-41d8-b7c4-a75ffa2067a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:50.528235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65535ec-0711-41d8-b7c4-a75ffa2067a3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:50.531817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53269b-bbb1-40e2-9c3f-b205a1a8217c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:50.532457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53269b-bbb1-40e2-9c3f-b205a1a8217c service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:50.532740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53269b-bbb1-40e2-9c3f-b205a1a8217c service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:50.533018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53269b-bbb1-40e2-9c3f-b205a1a8217c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:50.543659 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53269b-bbb1-40e2-9c3f-b205a1a8217c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:50.545128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53269b-bbb1-40e2-9c3f-b205a1a8217c service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:50.556233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 714/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:53.054231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657488e-271b-4287-a3cb-31058e3dd8a1 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:53.054815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657488e-271b-4287-a3cb-31058e3dd8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:53.054815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657488e-271b-4287-a3cb-31058e3dd8a1 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:53.056156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:53.066878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245f13d-8f4f-48f4-bf5d-5547d4acf8d5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:53.067441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245f13d-8f4f-48f4-bf5d-5547d4acf8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:53.067441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245f13d-8f4f-48f4-bf5d-5547d4acf8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:53.086253 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7245f13d-8f4f-48f4-bf5d-5547d4acf8d5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:53.087461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7245f13d-8f4f-48f4-bf5d-5547d4acf8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:53.095456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:53.135464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657488e-271b-4287-a3cb-31058e3dd8a1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:53.146318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657488e-271b-4287-a3cb-31058e3dd8a1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:53.150817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67470a-f08f-433d-8dcb-2161878efd3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:53.151462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67470a-f08f-433d-8dcb-2161878efd3b service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:53.151741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67470a-f08f-433d-8dcb-2161878efd3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:53.152017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67470a-f08f-433d-8dcb-2161878efd3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:53.161644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67470a-f08f-433d-8dcb-2161878efd3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:53.163708 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67470a-f08f-433d-8dcb-2161878efd3b service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:53.173566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 737/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:55.512018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403bf01-f9f1-42d8-9002-31ca4e9e2234 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:55.512594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403bf01-f9f1-42d8-9002-31ca4e9e2234 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:55.512917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403bf01-f9f1-42d8-9002-31ca4e9e2234 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:55.514858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:15:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:55.520186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c76dfa-26e9-49a1-a8b7-ac2769ab39ba None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:55.520622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c76dfa-26e9-49a1-a8b7-ac2769ab39ba None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:55.520908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c76dfa-26e9-49a1-a8b7-ac2769ab39ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:55.539846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a1c76dfa-26e9-49a1-a8b7-ac2769ab39ba None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:55.541049 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a1c76dfa-26e9-49a1-a8b7-ac2769ab39ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:55.553781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:55.591780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403bf01-f9f1-42d8-9002-31ca4e9e2234 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:55.605274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403bf01-f9f1-42d8-9002-31ca4e9e2234 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:55.609113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bef2b-207e-48de-b96d-cb50c6b9b26c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:55.609829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bef2b-207e-48de-b96d-cb50c6b9b26c service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:55.610120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bef2b-207e-48de-b96d-cb50c6b9b26c service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:55.610398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bef2b-207e-48de-b96d-cb50c6b9b26c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:55.620634 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bef2b-207e-48de-b96d-cb50c6b9b26c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:55.622139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bef2b-207e-48de-b96d-cb50c6b9b26c service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:55.632719 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 717/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:15:58.099246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92cf77-2887-4619-946c-230105b482d5 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:58.099845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92cf77-2887-4619-946c-230105b482d5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:58.099845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92cf77-2887-4619-946c-230105b482d5 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:58.101743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:15:58.107105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a24e80-453b-4445-9607-4902817b31a2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:58.107945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a24e80-453b-4445-9607-4902817b31a2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:58.107945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a24e80-453b-4445-9607-4902817b31a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:58.113861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=70 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:15:58.128004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-30a24e80-453b-4445-9607-4902817b31a2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:15:58.129178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-30a24e80-453b-4445-9607-4902817b31a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:15:58.137823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:15:58.176389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92cf77-2887-4619-946c-230105b482d5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:15:58.182609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=64 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:15:58.189130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92cf77-2887-4619-946c-230105b482d5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:15:58.192614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484056f2-a34d-4924-8419-2df0cc207cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:15:58.193274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484056f2-a34d-4924-8419-2df0cc207cbf service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:15:58.194253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484056f2-a34d-4924-8419-2df0cc207cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:15:58.194881 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484056f2-a34d-4924-8419-2df0cc207cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:15:58.205571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484056f2-a34d-4924-8419-2df0cc207cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:15:58.207089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484056f2-a34d-4924-8419-2df0cc207cbf service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:15:58.216986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 740/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:16:00.585997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1195dfc-f1e5-4e79-8c63-7356519f0c53 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:00.586576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1195dfc-f1e5-4e79-8c63-7356519f0c53 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:00.586682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1195dfc-f1e5-4e79-8c63-7356519f0c53 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:00.588839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:00.595716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6a33c-c1c9-472a-b146-54db2ed0850e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:00.596244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6a33c-c1c9-472a-b146-54db2ed0850e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:00.596244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6a33c-c1c9-472a-b146-54db2ed0850e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:00.614838 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-57c6a33c-c1c9-472a-b146-54db2ed0850e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:00.616422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-57c6a33c-c1c9-472a-b146-54db2ed0850e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:00.624257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:00.663205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1195dfc-f1e5-4e79-8c63-7356519f0c53 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:00.674234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1195dfc-f1e5-4e79-8c63-7356519f0c53 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:00.677899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26786bde-550e-4b41-b922-42a8d1f9d074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:00.678544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26786bde-550e-4b41-b922-42a8d1f9d074 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:00.678829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26786bde-550e-4b41-b922-42a8d1f9d074 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:00.679107 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26786bde-550e-4b41-b922-42a8d1f9d074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:00.688745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26786bde-550e-4b41-b922-42a8d1f9d074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:00.690218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26786bde-550e-4b41-b922-42a8d1f9d074 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:00.700865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 720/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:16:03.070186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565cbf8-a6cd-490e-a492-fa0d00a4f195 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:03.070785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565cbf8-a6cd-490e-a492-fa0d00a4f195 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:03.070785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565cbf8-a6cd-490e-a492-fa0d00a4f195 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:03.072144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:03.077730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995e701-6757-4720-9125-4918c168fa63 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:03.078162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995e701-6757-4720-9125-4918c168fa63 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:03.078349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995e701-6757-4720-9125-4918c168fa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:03.096192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a995e701-6757-4720-9125-4918c168fa63 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:03.097373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a995e701-6757-4720-9125-4918c168fa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:03.106058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:03.144120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565cbf8-a6cd-490e-a492-fa0d00a4f195 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:03.155287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565cbf8-a6cd-490e-a492-fa0d00a4f195 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:03.158853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6c844-4271-47fb-9b8c-9052ed12a47d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:03.159499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6c844-4271-47fb-9b8c-9052ed12a47d service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:03.159779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6c844-4271-47fb-9b8c-9052ed12a47d service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:03.160050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6c844-4271-47fb-9b8c-9052ed12a47d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:03.171277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6c844-4271-47fb-9b8c-9052ed12a47d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:03.172729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6c844-4271-47fb-9b8c-9052ed12a47d service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:03.183098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 743/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:16:05.546778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863a2ad-e6df-4ed1-8daa-073ad027a9c9 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:05.547372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863a2ad-e6df-4ed1-8daa-073ad027a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:05.547372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863a2ad-e6df-4ed1-8daa-073ad027a9c9 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:05.548730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:16:05.555124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e2caa-66db-47b0-87ab-fd5ae0f1df14 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:05.555606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e2caa-66db-47b0-87ab-fd5ae0f1df14 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:05.555705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e2caa-66db-47b0-87ab-fd5ae0f1df14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:05.574003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-723e2caa-66db-47b0-87ab-fd5ae0f1df14 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:05.575218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-723e2caa-66db-47b0-87ab-fd5ae0f1df14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:05.583360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:05.620546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863a2ad-e6df-4ed1-8daa-073ad027a9c9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:05.631807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863a2ad-e6df-4ed1-8daa-073ad027a9c9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:05.635340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4abd8-d430-4e25-8746-3c91d8e57f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:05.635977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4abd8-d430-4e25-8746-3c91d8e57f5e service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:05.636257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4abd8-d430-4e25-8746-3c91d8e57f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:05.636528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4abd8-d430-4e25-8746-3c91d8e57f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:05.645694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4abd8-d430-4e25-8746-3c91d8e57f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:05.647155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4abd8-d430-4e25-8746-3c91d8e57f5e service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:05.657330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 723/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:16:07.988525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06afbfb-9026-4c8e-b2c1-26dd72bd1fe3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:07.989107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06afbfb-9026-4c8e-b2c1-26dd72bd1fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:07.989107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06afbfb-9026-4c8e-b2c1-26dd72bd1fe3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:07.990907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:07.997071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131df07-a084-4e9a-94b3-d6d148780d3c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:07.997799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131df07-a084-4e9a-94b3-d6d148780d3c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:07.997799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131df07-a084-4e9a-94b3-d6d148780d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:08.017343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0131df07-a084-4e9a-94b3-d6d148780d3c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:08.018609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0131df07-a084-4e9a-94b3-d6d148780d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:08.027960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:08.065955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06afbfb-9026-4c8e-b2c1-26dd72bd1fe3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:08.077556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06afbfb-9026-4c8e-b2c1-26dd72bd1fe3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:08.081209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e88348-031b-477d-94ad-f6a2323db9d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:08.081905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e88348-031b-477d-94ad-f6a2323db9d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:08.082192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e88348-031b-477d-94ad-f6a2323db9d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:08.082469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e88348-031b-477d-94ad-f6a2323db9d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:08.092771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e88348-031b-477d-94ad-f6a2323db9d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:08.094275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e88348-031b-477d-94ad-f6a2323db9d6 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:08.104798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 746/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:16:18.011927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=20 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:16:18.099709 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=23 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:16:40.802911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb200f-69d3-4793-9849-a31ef2f84b46 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:40.803599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb200f-69d3-4793-9849-a31ef2f84b46 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:40.803599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb200f-69d3-4793-9849-a31ef2f84b46 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:40.804859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 725/1471] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:40.810685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae67ba0-cff4-4292-a3b0-ead49da26fc3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:40.811162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae67ba0-cff4-4292-a3b0-ead49da26fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:40.811279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae67ba0-cff4-4292-a3b0-ead49da26fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:40.829091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-dae67ba0-cff4-4292-a3b0-ead49da26fc3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:40.830328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dae67ba0-cff4-4292-a3b0-ead49da26fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:40.838703 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 747/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:40.889855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d2ad4-895c-46d3-879b-8561f00fd1f0 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:40.890199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d2ad4-895c-46d3-879b-8561f00fd1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:40.890657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d2ad4-895c-46d3-879b-8561f00fd1f0 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:40.892175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 726/1473] 10.222.0.13 () {54 vars in 962 bytes} [Mon Jun 6 14:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:40.898302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30151d67-3e52-4364-8590-5881deb2f1e8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:40.898588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30151d67-3e52-4364-8590-5881deb2f1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:40.898869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30151d67-3e52-4364-8590-5881deb2f1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:40.918179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-30151d67-3e52-4364-8590-5881deb2f1e8 None None] MFA Rules not processed for user `f41a357957bc47739b7bd6f21d159d74`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:40.919423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-30151d67-3e52-4364-8590-5881deb2f1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:40.926407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 748/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Mon Jun 6 14:16:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:40.934063 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d2ad4-895c-46d3-879b-8561f00fd1f0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:40.945268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d2ad4-895c-46d3-879b-8561f00fd1f0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:40.948903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ec856-2e51-4f2f-be0e-d3814f8b104f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41a357957bc47739b7bd6f21d159d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:40.949545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ec856-2e51-4f2f-be0e-d3814f8b104f service glance] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:40.949874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ec856-2e51-4f2f-be0e-d3814f8b104f service glance] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:40.950156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ec856-2e51-4f2f-be0e-d3814f8b104f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:40.960302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ec856-2e51-4f2f-be0e-d3814f8b104f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:40.961774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ec856-2e51-4f2f-be0e-d3814f8b104f service glance] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:40.971413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 727/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:16:43.385672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9b30f-8b43-42a1-856b-7b6288b7b6bd None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:43.386299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9b30f-8b43-42a1-856b-7b6288b7b6bd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:43.386619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9b30f-8b43-42a1-856b-7b6288b7b6bd None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:43.388852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 749/1476] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:16:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:43.394740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a317387-363a-4d85-b0a1-1273f1ed8044 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:43.396341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a317387-363a-4d85-b0a1-1273f1ed8044 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:43.396341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a317387-363a-4d85-b0a1-1273f1ed8044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:43.413828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3a317387-363a-4d85-b0a1-1273f1ed8044 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:43.415045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3a317387-363a-4d85-b0a1-1273f1ed8044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:43.423099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 728/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:43.448550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9b30f-8b43-42a1-856b-7b6288b7b6bd None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:43.461551 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9b30f-8b43-42a1-856b-7b6288b7b6bd None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:43.466005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6ce3a-efe1-4066-9e73-81f161b0f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:43.466652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6ce3a-efe1-4066-9e73-81f161b0f372 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:43.466934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6ce3a-efe1-4066-9e73-81f161b0f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:43.467208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6ce3a-efe1-4066-9e73-81f161b0f372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:43.467932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6ce3a-efe1-4066-9e73-81f161b0f372 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:43.469682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-30b6ce3a-efe1-4066-9e73-81f161b0f372 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:16:43.469682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: warnings.warn(msg) Jun 06 14:16:43.469682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: Jun 06 14:16:43.470832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6ce3a-efe1-4066-9e73-81f161b0f372 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:43.492182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 750/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 6 14:16:43 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:16:45.808075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474d84b-38d3-433e-becb-0a4354182210 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:45.808602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474d84b-38d3-433e-becb-0a4354182210 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:45.808602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474d84b-38d3-433e-becb-0a4354182210 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:45.809949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 729/1479] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:45.815233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685dc43e-19f6-474a-bc3a-efebc924e781 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:45.815666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685dc43e-19f6-474a-bc3a-efebc924e781 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:45.815775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685dc43e-19f6-474a-bc3a-efebc924e781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:45.833906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-685dc43e-19f6-474a-bc3a-efebc924e781 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:45.835124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-685dc43e-19f6-474a-bc3a-efebc924e781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:45.843224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 751/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:45.892846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474d84b-38d3-433e-becb-0a4354182210 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:45.907479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474d84b-38d3-433e-becb-0a4354182210 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:45.913878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f5523-f5fe-48c5-9c9a-e10e13f4f4fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:45.914526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f5523-f5fe-48c5-9c9a-e10e13f4f4fe service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:45.914813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f5523-f5fe-48c5-9c9a-e10e13f4f4fe service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:45.915098 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f5523-f5fe-48c5-9c9a-e10e13f4f4fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:45.923552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f5523-f5fe-48c5-9c9a-e10e13f4f4fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:45.925136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f5523-f5fe-48c5-9c9a-e10e13f4f4fe service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:45.935314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 730/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:16:48.399954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bd563-67ea-4239-8e9a-a8b8b8af87ac None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:48.400518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bd563-67ea-4239-8e9a-a8b8b8af87ac None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:48.400518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bd563-67ea-4239-8e9a-a8b8b8af87ac None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:48.401848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 752/1482] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:48.407301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e41d6-586b-4c68-96ed-b4816fde3c80 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:48.407727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e41d6-586b-4c68-96ed-b4816fde3c80 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:48.407839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e41d6-586b-4c68-96ed-b4816fde3c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:48.426606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7e5e41d6-586b-4c68-96ed-b4816fde3c80 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:48.427819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7e5e41d6-586b-4c68-96ed-b4816fde3c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:48.436572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 731/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:48.473797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bd563-67ea-4239-8e9a-a8b8b8af87ac None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:48.485722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bd563-67ea-4239-8e9a-a8b8b8af87ac None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:48.489791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8770345-c0d5-43de-b683-0ede9b20cb4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:48.490432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8770345-c0d5-43de-b683-0ede9b20cb4d service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:48.490719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8770345-c0d5-43de-b683-0ede9b20cb4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:48.490999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8770345-c0d5-43de-b683-0ede9b20cb4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:48.500185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8770345-c0d5-43de-b683-0ede9b20cb4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:48.501639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8770345-c0d5-43de-b683-0ede9b20cb4d service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:48.511909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 753/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:16:50.866759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6e37-6f7d-4323-9d06-ba3c872d7f8f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:50.867336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6e37-6f7d-4323-9d06-ba3c872d7f8f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:50.867336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6e37-6f7d-4323-9d06-ba3c872d7f8f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:50.868599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 732/1485] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:50.874200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f9b7e-2ab9-4c2f-a7d6-e500519380a2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:50.874658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f9b7e-2ab9-4c2f-a7d6-e500519380a2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:50.875090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f9b7e-2ab9-4c2f-a7d6-e500519380a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:50.894736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-524f9b7e-2ab9-4c2f-a7d6-e500519380a2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:50.896040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-524f9b7e-2ab9-4c2f-a7d6-e500519380a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:50.903955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 754/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:50.943035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6e37-6f7d-4323-9d06-ba3c872d7f8f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:50.953727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6e37-6f7d-4323-9d06-ba3c872d7f8f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:50.957833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc793f-6b13-4483-af2d-dddbf632241f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:50.958466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc793f-6b13-4483-af2d-dddbf632241f service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:50.958749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc793f-6b13-4483-af2d-dddbf632241f service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:50.959025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc793f-6b13-4483-af2d-dddbf632241f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:50.968122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc793f-6b13-4483-af2d-dddbf632241f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:50.969539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc793f-6b13-4483-af2d-dddbf632241f service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:50.979082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 733/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:16:53.404367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e092c9-e059-45d9-9b47-51b760236de3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:53.404969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e092c9-e059-45d9-9b47-51b760236de3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:53.404969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e092c9-e059-45d9-9b47-51b760236de3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:53.406374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 755/1488] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:53.411942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372bcc5-a17f-4742-9aed-2c8285940ea8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:53.412477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372bcc5-a17f-4742-9aed-2c8285940ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:53.412477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372bcc5-a17f-4742-9aed-2c8285940ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:53.430717 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8372bcc5-a17f-4742-9aed-2c8285940ea8 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:53.432458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8372bcc5-a17f-4742-9aed-2c8285940ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:53.439785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 734/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:16:53.480536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e092c9-e059-45d9-9b47-51b760236de3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:53.492355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e092c9-e059-45d9-9b47-51b760236de3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:53.495796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e42774-0284-49df-b20c-bc61cc5f8c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:53.496433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e42774-0284-49df-b20c-bc61cc5f8c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:53.496713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e42774-0284-49df-b20c-bc61cc5f8c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:53.496987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e42774-0284-49df-b20c-bc61cc5f8c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:53.507145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e42774-0284-49df-b20c-bc61cc5f8c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:53.508633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e42774-0284-49df-b20c-bc61cc5f8c93 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:53.518228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 756/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:16:56.028279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88ed5b-65d6-4018-8bd1-430907643301 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:56.028827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88ed5b-65d6-4018-8bd1-430907643301 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:56.028827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88ed5b-65d6-4018-8bd1-430907643301 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:56.030222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 735/1491] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 6 14:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:16:56.036333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cefacf-739e-4cae-9d10-8e0ba1746ea8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:56.037001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cefacf-739e-4cae-9d10-8e0ba1746ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:56.037001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cefacf-739e-4cae-9d10-8e0ba1746ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:56.054621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-58cefacf-739e-4cae-9d10-8e0ba1746ea8 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:16:56.055846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-58cefacf-739e-4cae-9d10-8e0ba1746ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:16:56.063778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 757/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 6 14:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:16:56.103186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88ed5b-65d6-4018-8bd1-430907643301 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:16:56.114206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88ed5b-65d6-4018-8bd1-430907643301 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:16:56.117789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e4497-1d7a-4751-8c21-afd4f53e7a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:16:56.118430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e4497-1d7a-4751-8c21-afd4f53e7a80 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:16:56.118715 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e4497-1d7a-4751-8c21-afd4f53e7a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:16:56.118991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e4497-1d7a-4751-8c21-afd4f53e7a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:16:56.128296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e4497-1d7a-4751-8c21-afd4f53e7a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:16:56.129742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e4497-1d7a-4751-8c21-afd4f53e7a80 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:16:56.139390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 736/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:17:06.049209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=33 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:17:06.135133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=33 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:18:24.981099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc2c20-67cf-4bf1-80bd-df9d226099e1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:24.981676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc2c20-67cf-4bf1-80bd-df9d226099e1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:24.981676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc2c20-67cf-4bf1-80bd-df9d226099e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.000262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ccbc2c20-67cf-4bf1-80bd-df9d226099e1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:18:25.001498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ccbc2c20-67cf-4bf1-80bd-df9d226099e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:18:25.009565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 758/1494] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:18:25.018517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f60bb-07c4-4ed4-8d7f-ccd3fa366f5f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.019025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f60bb-07c4-4ed4-8d7f-ccd3fa366f5f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.019129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f60bb-07c4-4ed4-8d7f-ccd3fa366f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.037808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6f8f60bb-07c4-4ed4-8d7f-ccd3fa366f5f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:18:25.038982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6f8f60bb-07c4-4ed4-8d7f-ccd3fa366f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:18:25.047410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 737/1495] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:18:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:18:25.051953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc2c20-67cf-4bf1-80bd-df9d226099e1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:25.062059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc2c20-67cf-4bf1-80bd-df9d226099e1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:25.067257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:25.067918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.068202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.068573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.069188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:18:25.069529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:25.070632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:18:25.070632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: warnings.warn(msg) Jun 06 14:18:25.070632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: Jun 06 14:18:25.071413 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:25.083774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 759/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:18:25.091897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f60bb-07c4-4ed4-8d7f-ccd3fa366f5f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:25.101667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f60bb-07c4-4ed4-8d7f-ccd3fa366f5f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:25.106370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290d9d9-9603-4373-bd77-8ef64f1a7783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:25.107017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290d9d9-9603-4373-bd77-8ef64f1a7783 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.107300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290d9d9-9603-4373-bd77-8ef64f1a7783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.107572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290d9d9-9603-4373-bd77-8ef64f1a7783 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.108785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290d9d9-9603-4373-bd77-8ef64f1a7783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:25.110905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-0290d9d9-9603-4373-bd77-8ef64f1a7783 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:18:25.110905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jun 06 14:18:25.110905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: Jun 06 14:18:25.112085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290d9d9-9603-4373-bd77-8ef64f1a7783 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:25.150267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 738/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:18:25 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:18:25.154376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:25.164842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4f2e-34ce-4226-81b0-03b424f48520 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:25.222069 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9a92e-7d15-4f98-b684-a9499b7a44bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:25.222745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a92e-7d15-4f98-b684-a9499b7a44bb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.223033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a92e-7d15-4f98-b684-a9499b7a44bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.223310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a92e-7d15-4f98-b684-a9499b7a44bb admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.224014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9a92e-7d15-4f98-b684-a9499b7a44bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:25.225036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-fac9a92e-7d15-4f98-b684-a9499b7a44bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:18:25.225036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: warnings.warn(msg) Jun 06 14:18:25.225036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: Jun 06 14:18:25.226289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9a92e-7d15-4f98-b684-a9499b7a44bb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:25.249338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 760/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:18:25 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:18:25.253870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290d9d9-9603-4373-bd77-8ef64f1a7783 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:25.264093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290d9d9-9603-4373-bd77-8ef64f1a7783 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:25.268614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94872443-4496-4b53-9b53-841966ceabdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:25.269277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94872443-4496-4b53-9b53-841966ceabdd admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.269560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94872443-4496-4b53-9b53-841966ceabdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.269864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94872443-4496-4b53-9b53-841966ceabdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.270616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94872443-4496-4b53-9b53-841966ceabdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:25.271619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-94872443-4496-4b53-9b53-841966ceabdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:18:25.271619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jun 06 14:18:25.271619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: Jun 06 14:18:25.272501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94872443-4496-4b53-9b53-841966ceabdd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:25.287796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 739/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:18:25.291972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9a92e-7d15-4f98-b684-a9499b7a44bb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:25.302528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9a92e-7d15-4f98-b684-a9499b7a44bb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:25.307125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c727a7f-efb6-49a4-9144-09fb7a8c501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:25.307786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c727a7f-efb6-49a4-9144-09fb7a8c501d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.308070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c727a7f-efb6-49a4-9144-09fb7a8c501d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.308348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c727a7f-efb6-49a4-9144-09fb7a8c501d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.309074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c727a7f-efb6-49a4-9144-09fb7a8c501d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:25.310124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-5c727a7f-efb6-49a4-9144-09fb7a8c501d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:18:25.310124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: warnings.warn(msg) Jun 06 14:18:25.310124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: Jun 06 14:18:25.311017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c727a7f-efb6-49a4-9144-09fb7a8c501d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:25.325390 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 761/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:18:25.330149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94872443-4496-4b53-9b53-841966ceabdd admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:25.340358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94872443-4496-4b53-9b53-841966ceabdd admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:25.345589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b2c21-c273-4659-8203-6b3be7cb46ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:25.346491 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b2c21-c273-4659-8203-6b3be7cb46ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.346774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b2c21-c273-4659-8203-6b3be7cb46ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.347055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b2c21-c273-4659-8203-6b3be7cb46ac admin admin] PATH_INFO: `/v3/projects/fb872483edc642129cd28baf8bcaa993/users/ce38fb5f85ed44ae9bd1052d9809b07a/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.363319 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b2c21-c273-4659-8203-6b3be7cb46ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb872483edc642129cd28baf8bcaa993, user_id=ce38fb5f85ed44ae9bd1052d9809b07a, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:25.364345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-6d2b2c21-c273-4659-8203-6b3be7cb46ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:18:25.364345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jun 06 14:18:25.364345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: Jun 06 14:18:25.366313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b2c21-c273-4659-8203-6b3be7cb46ac admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:25.380541 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 740/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:18:25 2022] PUT /identity/v3/projects/fb872483edc642129cd28baf8bcaa993/users/ce38fb5f85ed44ae9bd1052d9809b07a/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:18:25.393373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b86fd-fab3-4a9b-a80c-4a2dbbb93531 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.393681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b86fd-fab3-4a9b-a80c-4a2dbbb93531 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.393998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b86fd-fab3-4a9b-a80c-4a2dbbb93531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.420295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cf9b86fd-fab3-4a9b-a80c-4a2dbbb93531 None None] MFA Rules not processed for user `ce38fb5f85ed44ae9bd1052d9809b07a`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:18:25.422171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cf9b86fd-fab3-4a9b-a80c-4a2dbbb93531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:18:25.488981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 762/1502] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:18:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:18:25.502422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060430d7-5f9d-4fc2-a707-2dc8a41ad9a6 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.502636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060430d7-5f9d-4fc2-a707-2dc8a41ad9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.502961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060430d7-5f9d-4fc2-a707-2dc8a41ad9a6 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.504335 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 741/1503] 10.222.0.13 () {54 vars in 963 bytes} [Mon Jun 6 14:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:18:25.509642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbc7a9-bd8b-497d-ab69-134e868bf666 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.509985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbc7a9-bd8b-497d-ab69-134e868bf666 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.510263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbc7a9-bd8b-497d-ab69-134e868bf666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.529263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-01dbc7a9-bd8b-497d-ab69-134e868bf666 None None] MFA Rules not processed for user `720c995318a24e7b859fa82febd8178b`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:18:25.531559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-01dbc7a9-bd8b-497d-ab69-134e868bf666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:18:25.577823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:18:25.586691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 763/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Mon Jun 6 14:18:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:18:25.594465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060430d7-5f9d-4fc2-a707-2dc8a41ad9a6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:25.605022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060430d7-5f9d-4fc2-a707-2dc8a41ad9a6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:25.609053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe770b-e14c-438a-a7c4-621e8f44bfff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:25.609694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe770b-e14c-438a-a7c4-621e8f44bfff service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.610010 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe770b-e14c-438a-a7c4-621e8f44bfff service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.610287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe770b-e14c-438a-a7c4-621e8f44bfff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.618930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe770b-e14c-438a-a7c4-621e8f44bfff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:25.620361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe770b-e14c-438a-a7c4-621e8f44bfff service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:25.630556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 742/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:18:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:18:25.673321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b66c4e-cfa7-441e-b3e6-216eafff14a0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.673615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b66c4e-cfa7-441e-b3e6-216eafff14a0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.673949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b66c4e-cfa7-441e-b3e6-216eafff14a0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.675244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 764/1506] 10.222.0.13 () {52 vars in 829 bytes} [Mon Jun 6 14:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:18:25.678926 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe770b-e14c-438a-a7c4-621e8f44bfff service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:25.751681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe770b-e14c-438a-a7c4-621e8f44bfff service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:25.756522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:25.757346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.757632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.757935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] PATH_INFO: `/v3/users/ce38fb5f85ed44ae9bd1052d9809b07a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.759156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce38fb5f85ed44ae9bd1052d9809b07a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:25.760210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:18:25.760210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jun 06 14:18:25.760210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: Jun 06 14:18:25.761420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:25.830825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce38fb5f85ed44ae9bd1052d9809b07a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:25.843857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce38fb5f85ed44ae9bd1052d9809b07a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:25.844730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce38fb5f85ed44ae9bd1052d9809b07a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:25.864403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce38fb5f85ed44ae9bd1052d9809b07a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:25.865155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce38fb5f85ed44ae9bd1052d9809b07a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:25.869163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:18:25.882610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 743/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:18:25 2022] DELETE /identity/v3/users/ce38fb5f85ed44ae9bd1052d9809b07a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:18:25.887063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8bd00-f9d8-4337-9775-4be291369f34 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:25.887357 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8bd00-f9d8-4337-9775-4be291369f34 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:25.887650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8bd00-f9d8-4337-9775-4be291369f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:25.905097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-62b8bd00-f9d8-4337-9775-4be291369f34 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:18:25.906458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-62b8bd00-f9d8-4337-9775-4be291369f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:18:25.966611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 765/1508] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:18:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:18:25.976916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:25.978347 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:18:26.039150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a5c86-a0dd-44a9-aae2-ae7afc5e2b80 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:26.042889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47142cd9-3fff-4b6e-b4c9-ffbcf8f24877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:26.043550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47142cd9-3fff-4b6e-b4c9-ffbcf8f24877 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:26.043830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47142cd9-3fff-4b6e-b4c9-ffbcf8f24877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:26.044103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47142cd9-3fff-4b6e-b4c9-ffbcf8f24877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:26.054149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47142cd9-3fff-4b6e-b4c9-ffbcf8f24877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:26.055583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47142cd9-3fff-4b6e-b4c9-ffbcf8f24877 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:26.065777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 744/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:18:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:18:26.401192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8bd00-f9d8-4337-9775-4be291369f34 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:18:26.402703 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-62b8bd00-f9d8-4337-9775-4be291369f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:18:26.415173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8bd00-f9d8-4337-9775-4be291369f34 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:18:26.420036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:18:26.421550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:18:26.421875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:18:26.422155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] PATH_INFO: `/v3/projects/fb872483edc642129cd28baf8bcaa993` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:18:26.423499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb872483edc642129cd28baf8bcaa993)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:18:26.424541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:18:26.424541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: warnings.warn(msg) Jun 06 14:18:26.424541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: Jun 06 14:18:26.425784 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:18:26.494303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb872483edc642129cd28baf8bcaa993', 'request_id': 'req-cbf818ae-24dd-44f0-8c50-eac05e2172a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:26.506683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb872483edc642129cd28baf8bcaa993', 'request_id': 'req-cbf818ae-24dd-44f0-8c50-eac05e2172a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:26.521680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb872483edc642129cd28baf8bcaa993', 'request_id': 'req-cbf818ae-24dd-44f0-8c50-eac05e2172a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:26.533785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] The token cache is being invalidate because project fb872483edc642129cd28baf8bcaa993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:18:26.534075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:26.534743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cbf818ae-24dd-44f0-8c50-eac05e2172a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:18:26.536386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 766/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:18:26 2022] DELETE /identity/v3/projects/fb872483edc642129cd28baf8bcaa993 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:18:36.061328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:18:36.529579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:20:04.674592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142d55b-5262-4f14-9b23-469144679c19 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:04.675205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142d55b-5262-4f14-9b23-469144679c19 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:04.675205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142d55b-5262-4f14-9b23-469144679c19 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:04.676533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:20:04.681611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8513a3f-98f5-40f1-857b-5889ac76c5be None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:04.682337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8513a3f-98f5-40f1-857b-5889ac76c5be None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:04.682572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8513a3f-98f5-40f1-857b-5889ac76c5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:04.701269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d8513a3f-98f5-40f1-857b-5889ac76c5be None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:04.702505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d8513a3f-98f5-40f1-857b-5889ac76c5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:04.762001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:20:04.936376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142d55b-5262-4f14-9b23-469144679c19 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:20:04.938538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2142d55b-5262-4f14-9b23-469144679c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:05.028746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142d55b-5262-4f14-9b23-469144679c19 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:20:05.034740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b8dc5-263e-46cc-b57d-5d6ec84af75f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:20:05.035390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8dc5-263e-46cc-b57d-5d6ec84af75f service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:05.035673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8dc5-263e-46cc-b57d-5d6ec84af75f service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:05.035950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8dc5-263e-46cc-b57d-5d6ec84af75f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:05.044110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8dc5-263e-46cc-b57d-5d6ec84af75f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:20:05.045682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8dc5-263e-46cc-b57d-5d6ec84af75f service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:20:05.054993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 746/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:20:05.807979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff5a4b-440c-4537-bdfd-4f26c03d19d5 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:05.808529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff5a4b-440c-4537-bdfd-4f26c03d19d5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:05.808641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff5a4b-440c-4537-bdfd-4f26c03d19d5 None None] PATH_INFO: `/v3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:05.809894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 768/1514] 10.222.0.46 () {54 vars in 925 bytes} [Mon Jun 6 14:20:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:20:05.835535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf594ed1-1ae1-4051-8916-b59d03b2545e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:05.836042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf594ed1-1ae1-4051-8916-b59d03b2545e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:05.836167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf594ed1-1ae1-4051-8916-b59d03b2545e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:05.857023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-cf594ed1-1ae1-4051-8916-b59d03b2545e None None] MFA Rules not processed for user `51b91456a0634e0fa22fb389f0721a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:05.858251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cf594ed1-1ae1-4051-8916-b59d03b2545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:05.923906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 747/1515] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Jun 6 14:20:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:20:05.965213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fd065-31a2-463f-9cb3-f818b94434d2 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:05.965550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fd065-31a2-463f-9cb3-f818b94434d2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:05.965871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fd065-31a2-463f-9cb3-f818b94434d2 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:05.967270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 769/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:20:05.972751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b029c-1527-4bd2-9b39-d1fcc349f090 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:05.973038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b029c-1527-4bd2-9b39-d1fcc349f090 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:05.973342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b029c-1527-4bd2-9b39-d1fcc349f090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:05.993118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-cb1b029c-1527-4bd2-9b39-d1fcc349f090 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:05.994347 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cb1b029c-1527-4bd2-9b39-d1fcc349f090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:06.002305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 748/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:20:06.154976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fd065-31a2-463f-9cb3-f818b94434d2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:20:06.169748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fd065-31a2-463f-9cb3-f818b94434d2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:20:06.175732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec8100-9929-4e65-b042-b02be480455b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:20:06.176365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec8100-9929-4e65-b042-b02be480455b service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:06.176648 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec8100-9929-4e65-b042-b02be480455b service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:06.176923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec8100-9929-4e65-b042-b02be480455b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:06.185183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec8100-9929-4e65-b042-b02be480455b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:20:06.186849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec8100-9929-4e65-b042-b02be480455b service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:20:06.196576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 770/1518] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:20:09.915034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68501b3-f4aa-43ee-b43e-2af1f9829b01 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:09.915616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68501b3-f4aa-43ee-b43e-2af1f9829b01 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:09.915616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68501b3-f4aa-43ee-b43e-2af1f9829b01 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:09.916937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 749/1519] 10.222.0.13 () {54 vars in 960 bytes} [Mon Jun 6 14:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:20:09.922602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c554e0-d7c2-457c-a602-09ba87126a1a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:09.923121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c554e0-d7c2-457c-a602-09ba87126a1a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:09.923121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c554e0-d7c2-457c-a602-09ba87126a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:09.943216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-41c554e0-d7c2-457c-a602-09ba87126a1a None None] MFA Rules not processed for user `51b91456a0634e0fa22fb389f0721a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:09.944543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-41c554e0-d7c2-457c-a602-09ba87126a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:09.953441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 771/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Mon Jun 6 14:20:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:20:09.961233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68501b3-f4aa-43ee-b43e-2af1f9829b01 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:20:09.974612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68501b3-f4aa-43ee-b43e-2af1f9829b01 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:20:09.979513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9beb710-87df-475c-8b96-adfd31fc1809 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:20:09.980389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beb710-87df-475c-8b96-adfd31fc1809 service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:09.980678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beb710-87df-475c-8b96-adfd31fc1809 service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:09.980952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beb710-87df-475c-8b96-adfd31fc1809 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:09.989827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9beb710-87df-475c-8b96-adfd31fc1809 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:20:09.991399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9beb710-87df-475c-8b96-adfd31fc1809 service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:20:10.001664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 750/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:20:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:20:17.505202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486c1c7-7bf0-46e8-aacf-1d6ff72d980d None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:17.505782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486c1c7-7bf0-46e8-aacf-1d6ff72d980d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:17.505782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486c1c7-7bf0-46e8-aacf-1d6ff72d980d None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:17.507157 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:20:17.512183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e6bd6-bb05-4dbd-b076-faab5868b2f2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:17.512642 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e6bd6-bb05-4dbd-b076-faab5868b2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:17.512752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e6bd6-bb05-4dbd-b076-faab5868b2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:17.531871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3f6e6bd6-bb05-4dbd-b076-faab5868b2f2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:17.533031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3f6e6bd6-bb05-4dbd-b076-faab5868b2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:17.540878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:20:17.697065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486c1c7-7bf0-46e8-aacf-1d6ff72d980d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:20:17.712367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486c1c7-7bf0-46e8-aacf-1d6ff72d980d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:20:17.717779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0ff51-58bc-4f1a-ace2-850856cae2cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:20:17.719391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0ff51-58bc-4f1a-ace2-850856cae2cd service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:17.719523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0ff51-58bc-4f1a-ace2-850856cae2cd service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:17.719769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0ff51-58bc-4f1a-ace2-850856cae2cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:17.727944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0ff51-58bc-4f1a-ace2-850856cae2cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:20:17.729550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0ff51-58bc-4f1a-ace2-850856cae2cd service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:20:17.740166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 773/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:20:18.619166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3541f3-28fb-4faf-b9ed-0a5a9c766fd2 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:18.619767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3541f3-28fb-4faf-b9ed-0a5a9c766fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:18.619767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3541f3-28fb-4faf-b9ed-0a5a9c766fd2 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:18.621153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 14:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:20:18.626241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719763b-9c72-43ea-8111-1a3a4927f76a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:18.626733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719763b-9c72-43ea-8111-1a3a4927f76a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:18.626832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719763b-9c72-43ea-8111-1a3a4927f76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:18.646299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d719763b-9c72-43ea-8111-1a3a4927f76a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:18.647517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d719763b-9c72-43ea-8111-1a3a4927f76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:18.655324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 14:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:20:18.811051 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3541f3-28fb-4faf-b9ed-0a5a9c766fd2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:20:18.830762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3541f3-28fb-4faf-b9ed-0a5a9c766fd2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:20:18.836470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3b89-ff04-40e5-a512-f23699f42749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:20:18.837117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3b89-ff04-40e5-a512-f23699f42749 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:18.837399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3b89-ff04-40e5-a512-f23699f42749 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:18.837674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3b89-ff04-40e5-a512-f23699f42749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:18.848095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e3b89-ff04-40e5-a512-f23699f42749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:20:18.849671 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e3b89-ff04-40e5-a512-f23699f42749 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:20:18.860586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 753/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:20:22.926492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624e597-493f-4344-95d7-0ff3e7acfbb4 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:22.927137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624e597-493f-4344-95d7-0ff3e7acfbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:22.927137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624e597-493f-4344-95d7-0ff3e7acfbb4 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:22.928330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:20:22.933657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4733e6-d330-4a2a-903c-ad918d55f6b5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:22.934222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4733e6-d330-4a2a-903c-ad918d55f6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:22.934222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4733e6-d330-4a2a-903c-ad918d55f6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:22.954890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-cb4733e6-d330-4a2a-903c-ad918d55f6b5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:22.956121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cb4733e6-d330-4a2a-903c-ad918d55f6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:22.964128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:20:23.004206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624e597-493f-4344-95d7-0ff3e7acfbb4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:20:23.006035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5624e597-493f-4344-95d7-0ff3e7acfbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:23.070119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624e597-493f-4344-95d7-0ff3e7acfbb4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:20:23.073960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33699e-1b6b-4a7a-9f28-b885020b0ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:20:23.074634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33699e-1b6b-4a7a-9f28-b885020b0ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:23.074919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33699e-1b6b-4a7a-9f28-b885020b0ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:23.075193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33699e-1b6b-4a7a-9f28-b885020b0ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:23.086210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33699e-1b6b-4a7a-9f28-b885020b0ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:20:23.087677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33699e-1b6b-4a7a-9f28-b885020b0ee0 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:20:23.098980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 776/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:20:25.371590 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101663b-2118-497b-b417-4aa19e3d554b None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:25.372127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101663b-2118-497b-b417-4aa19e3d554b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:25.372127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101663b-2118-497b-b417-4aa19e3d554b None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:25.373423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:20:25.379026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7a0ed-9723-4dce-abd2-7981a27bd6f1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:25.379462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7a0ed-9723-4dce-abd2-7981a27bd6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:25.379601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7a0ed-9723-4dce-abd2-7981a27bd6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:25.398439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-96d7a0ed-9723-4dce-abd2-7981a27bd6f1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:25.399619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-96d7a0ed-9723-4dce-abd2-7981a27bd6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:25.408201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:20:25.443390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101663b-2118-497b-b417-4aa19e3d554b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:20:25.445039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b101663b-2118-497b-b417-4aa19e3d554b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:25.458584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101663b-2118-497b-b417-4aa19e3d554b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:20:25.462045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c809b22-2380-4768-bec6-fd4da25cfe7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:20:25.462669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c809b22-2380-4768-bec6-fd4da25cfe7a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:25.462948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c809b22-2380-4768-bec6-fd4da25cfe7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:25.463221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c809b22-2380-4768-bec6-fd4da25cfe7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:25.472877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c809b22-2380-4768-bec6-fd4da25cfe7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:20:25.474318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c809b22-2380-4768-bec6-fd4da25cfe7a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:20:25.483847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 756/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:20:29.239163 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19048a07-2bff-4489-ad3a-d1cd988b3d88 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:29.239724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19048a07-2bff-4489-ad3a-d1cd988b3d88 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:29.239724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19048a07-2bff-4489-ad3a-d1cd988b3d88 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:29.241143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:20:29.246408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629d06e-5138-4128-a1e6-362e3a61198f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:29.246839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629d06e-5138-4128-a1e6-362e3a61198f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:29.246960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629d06e-5138-4128-a1e6-362e3a61198f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:29.253485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:20:29.266885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6629d06e-5138-4128-a1e6-362e3a61198f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:29.268057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6629d06e-5138-4128-a1e6-362e3a61198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:29.275869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:20:29.548280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19048a07-2bff-4489-ad3a-d1cd988b3d88 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:20:29.549981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-19048a07-2bff-4489-ad3a-d1cd988b3d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:29.555710 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=75 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:20:29.608831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19048a07-2bff-4489-ad3a-d1cd988b3d88 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:20:29.612353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912b186-e806-4dc6-90eb-23df8af4744a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:20:29.612993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b186-e806-4dc6-90eb-23df8af4744a service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:29.613277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b186-e806-4dc6-90eb-23df8af4744a service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:29.613554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b186-e806-4dc6-90eb-23df8af4744a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:29.622418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912b186-e806-4dc6-90eb-23df8af4744a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:20:29.624376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912b186-e806-4dc6-90eb-23df8af4744a service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:20:29.633854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 779/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:20:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:20:31.909303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef087a-ca78-4454-a994-64f27781c245 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:31.909835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef087a-ca78-4454-a994-64f27781c245 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:31.909835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef087a-ca78-4454-a994-64f27781c245 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:31.911217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 14:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:20:31.916348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1fbc8-06b2-466d-8af0-1f669446274d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:31.916762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1fbc8-06b2-466d-8af0-1f669446274d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:31.916902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1fbc8-06b2-466d-8af0-1f669446274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:31.935337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-edd1fbc8-06b2-466d-8af0-1f669446274d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:20:31.936611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-edd1fbc8-06b2-466d-8af0-1f669446274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:31.944363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 14:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:20:32.216760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef087a-ca78-4454-a994-64f27781c245 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:20:32.218775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-feef087a-ca78-4454-a994-64f27781c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:20:32.231837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef087a-ca78-4454-a994-64f27781c245 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:20:32.236150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c6ad5-4e1f-4827-9691-a8598282109a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:20:32.236790 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c6ad5-4e1f-4827-9691-a8598282109a service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:20:32.237069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c6ad5-4e1f-4827-9691-a8598282109a service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:20:32.237343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c6ad5-4e1f-4827-9691-a8598282109a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:20:32.246640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c6ad5-4e1f-4827-9691-a8598282109a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:20:32.248043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c6ad5-4e1f-4827-9691-a8598282109a service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:20:32.259346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 759/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:20:41.930047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=28 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:20:42.255257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=8 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:07.120114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f1f6d-b264-433f-a3ba-738f661915aa None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:07.126771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f1f6d-b264-433f-a3ba-738f661915aa None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:07.127932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f1f6d-b264-433f-a3ba-738f661915aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:07.184168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-069f1f6d-b264-433f-a3ba-738f661915aa None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:07.192255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-069f1f6d-b264-433f-a3ba-738f661915aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:07.222947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 781/1540] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:07.243425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514c390-101b-49b8-8cb6-592babd36768 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:07.244270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514c390-101b-49b8-8cb6-592babd36768 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:07.244270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514c390-101b-49b8-8cb6-592babd36768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:07.298059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6514c390-101b-49b8-8cb6-592babd36768 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:07.299559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6514c390-101b-49b8-8cb6-592babd36768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:07.320785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 760/1541] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:07.328337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f1f6d-b264-433f-a3ba-738f661915aa None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:07.372962 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f1f6d-b264-433f-a3ba-738f661915aa None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:07.390660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef59304-5ced-42d2-b8a4-616228d0ac17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:07.390660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef59304-5ced-42d2-b8a4-616228d0ac17 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:07.391118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef59304-5ced-42d2-b8a4-616228d0ac17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:07.391954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef59304-5ced-42d2-b8a4-616228d0ac17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:07.394189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef59304-5ced-42d2-b8a4-616228d0ac17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:07.394347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef59304-5ced-42d2-b8a4-616228d0ac17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:07.402119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef59304-5ced-42d2-b8a4-616228d0ac17 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:07.448651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 782/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:07.459373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514c390-101b-49b8-8cb6-592babd36768 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:07.488131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514c390-101b-49b8-8cb6-592babd36768 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:07.500606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e9613-43a0-4e78-a6da-b98140942cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:07.512201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e9613-43a0-4e78-a6da-b98140942cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:07.512702 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e9613-43a0-4e78-a6da-b98140942cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:07.513204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e9613-43a0-4e78-a6da-b98140942cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:07.519092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e9613-43a0-4e78-a6da-b98140942cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:07.523064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e9613-43a0-4e78-a6da-b98140942cd1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:07.592642 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 761/1543] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:07 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:07.605773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef59304-5ced-42d2-b8a4-616228d0ac17 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:07.608447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d062f-d20a-4af3-b8ec-3069b0d2105d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:07.608955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d062f-d20a-4af3-b8ec-3069b0d2105d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:07.609437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d062f-d20a-4af3-b8ec-3069b0d2105d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:07.645397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef59304-5ced-42d2-b8a4-616228d0ac17 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:07.658489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-132d062f-d20a-4af3-b8ec-3069b0d2105d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:07.660652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-132d062f-d20a-4af3-b8ec-3069b0d2105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:07.875440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a39879-f236-425c-bacd-66906b65fa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:07.876582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a39879-f236-425c-bacd-66906b65fa9c admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:07.876871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a39879-f236-425c-bacd-66906b65fa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:07.877152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a39879-f236-425c-bacd-66906b65fa9c admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:07.877907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a39879-f236-425c-bacd-66906b65fa9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:07.880269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a39879-f236-425c-bacd-66906b65fa9c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:07.910206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 762/1544] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:07.920670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 783/1545] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:07 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:07.924863 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d062f-d20a-4af3-b8ec-3069b0d2105d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:07.929357 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f99ef0-ac8a-499b-97b1-c622580969d6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:07.929755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f99ef0-ac8a-499b-97b1-c622580969d6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:07.929947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f99ef0-ac8a-499b-97b1-c622580969d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:07.947823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d062f-d20a-4af3-b8ec-3069b0d2105d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:07.963585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-69f99ef0-ac8a-499b-97b1-c622580969d6 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:07.965430 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-69f99ef0-ac8a-499b-97b1-c622580969d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:07.971103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d337c-1a50-4477-b1d8-9dff81189fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:07.972337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d337c-1a50-4477-b1d8-9dff81189fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:07.972838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d337c-1a50-4477-b1d8-9dff81189fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:07.973378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d337c-1a50-4477-b1d8-9dff81189fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:07.974816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d337c-1a50-4477-b1d8-9dff81189fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:07.978225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d337c-1a50-4477-b1d8-9dff81189fbc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:07.981752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 784/1546] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:07.983776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59885-5ccb-4fbb-829e-26d64ebf6152 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:07.984055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59885-5ccb-4fbb-829e-26d64ebf6152 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:07.984335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59885-5ccb-4fbb-829e-26d64ebf6152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.005051 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 763/1547] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:08.006540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d337c-1a50-4477-b1d8-9dff81189fbc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.022172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a4b59885-5ccb-4fbb-829e-26d64ebf6152 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.023467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a4b59885-5ccb-4fbb-829e-26d64ebf6152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.043068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d337c-1a50-4477-b1d8-9dff81189fbc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.046051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 785/1548] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.047025 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b59885-5ccb-4fbb-829e-26d64ebf6152 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.066914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.067953 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.068207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b59885-5ccb-4fbb-829e-26d64ebf6152 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.068414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.068860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.070237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:08.070718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.072687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:22:08.072687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jun 06 14:22:08.072687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: Jun 06 14:22:08.075063 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.082458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dc931-3589-42e0-bb3c-11cabdb29733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.083174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dc931-3589-42e0-bb3c-11cabdb29733 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.083458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dc931-3589-42e0-bb3c-11cabdb29733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.084336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dc931-3589-42e0-bb3c-11cabdb29733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.084546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dc931-3589-42e0-bb3c-11cabdb29733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.086266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dc931-3589-42e0-bb3c-11cabdb29733 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.109560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 786/1549] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:08.111232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32c4a-74b6-4883-8514-bb679ae2b628 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.111529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32c4a-74b6-4883-8514-bb679ae2b628 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.111809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32c4a-74b6-4883-8514-bb679ae2b628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.117554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 764/1550] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:08.122321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.146287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f0b32c4a-74b6-4883-8514-bb679ae2b628 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.147564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f0b32c4a-74b6-4883-8514-bb679ae2b628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.158078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413d29b-798b-441c-b0bc-e13aa9c916b4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.173173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 787/1551] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.173900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b32c4a-74b6-4883-8514-bb679ae2b628 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.187481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a227-48cf-41c5-9aa1-f0acfe8602e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.189102 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a227-48cf-41c5-9aa1-f0acfe8602e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.189876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a227-48cf-41c5-9aa1-f0acfe8602e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.190348 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a227-48cf-41c5-9aa1-f0acfe8602e5 admin admin] PATH_INFO: `/v3/projects/f185078fd436428e818d6dba472bacb8/users/53505ac8ab6142a6bf0c3d6fe1cc721f/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.194873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b32c4a-74b6-4883-8514-bb679ae2b628 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.212367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e15062-3611-486f-be25-70e922d99c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.212745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e15062-3611-486f-be25-70e922d99c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.218003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e15062-3611-486f-be25-70e922d99c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.218003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e15062-3611-486f-be25-70e922d99c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.218003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e15062-3611-486f-be25-70e922d99c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.224578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-01e15062-3611-486f-be25-70e922d99c66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:22:08.224578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: warnings.warn(msg) Jun 06 14:22:08.224578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: Jun 06 14:22:08.229190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e15062-3611-486f-be25-70e922d99c66 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.276906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a227-48cf-41c5-9aa1-f0acfe8602e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f185078fd436428e818d6dba472bacb8, user_id=53505ac8ab6142a6bf0c3d6fe1cc721f, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.282420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a227-48cf-41c5-9aa1-f0acfe8602e5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.313283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 788/1552] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:08.317270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea6e73-d7cb-4b9c-a754-70c431c9a64a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.318152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea6e73-d7cb-4b9c-a754-70c431c9a64a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.318845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea6e73-d7cb-4b9c-a754-70c431c9a64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.342301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 765/1553] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:08 2022] PUT /identity/v3/projects/f185078fd436428e818d6dba472bacb8/users/53505ac8ab6142a6bf0c3d6fe1cc721f/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:22:08.343978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a227-48cf-41c5-9aa1-f0acfe8602e5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.386213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-96ea6e73-d7cb-4b9c-a754-70c431c9a64a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.389639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-96ea6e73-d7cb-4b9c-a754-70c431c9a64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.514340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 789/1554] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.522353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07072924-4988-4428-95e2-635abc0f4879 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.523110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07072924-4988-4428-95e2-635abc0f4879 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.523616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07072924-4988-4428-95e2-635abc0f4879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.528516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a227-48cf-41c5-9aa1-f0acfe8602e5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.538435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559332cf-c216-46ba-a576-bf4df97be822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.539646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559332cf-c216-46ba-a576-bf4df97be822 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.540227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559332cf-c216-46ba-a576-bf4df97be822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.540989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559332cf-c216-46ba-a576-bf4df97be822 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.543867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559332cf-c216-46ba-a576-bf4df97be822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:08.544643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559332cf-c216-46ba-a576-bf4df97be822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.554048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559332cf-c216-46ba-a576-bf4df97be822 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.564176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-07072924-4988-4428-95e2-635abc0f4879 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.567430 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-07072924-4988-4428-95e2-635abc0f4879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.576831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 790/1555] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.578131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07072924-4988-4428-95e2-635abc0f4879 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.578374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 766/1556] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:08.581741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43323c-852b-43dd-b8c3-133e208959c8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.581741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43323c-852b-43dd-b8c3-133e208959c8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.581741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43323c-852b-43dd-b8c3-133e208959c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.588917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07072924-4988-4428-95e2-635abc0f4879 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.595508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b0be8-0ff7-4ce9-976c-36ac49b7dfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.596216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b0be8-0ff7-4ce9-976c-36ac49b7dfec admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.596494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b0be8-0ff7-4ce9-976c-36ac49b7dfec admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.596775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b0be8-0ff7-4ce9-976c-36ac49b7dfec admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.597482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b0be8-0ff7-4ce9-976c-36ac49b7dfec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.599513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b0be8-0ff7-4ce9-976c-36ac49b7dfec admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.603603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4c43323c-852b-43dd-b8c3-133e208959c8 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.605643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4c43323c-852b-43dd-b8c3-133e208959c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.612122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:08.617810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 767/1557] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.617810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c72f78-0720-44bf-be05-eee3feac8e40 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.617810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c72f78-0720-44bf-be05-eee3feac8e40 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.617810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c72f78-0720-44bf-be05-eee3feac8e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.642501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 791/1558] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/users => generated 284 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:08.642501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43a7d-fde6-4101-b7e7-1efa40a1a616 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.642501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43a7d-fde6-4101-b7e7-1efa40a1a616 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.642501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43a7d-fde6-4101-b7e7-1efa40a1a616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.654605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-96c72f78-0720-44bf-be05-eee3feac8e40 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.656592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-96c72f78-0720-44bf-be05-eee3feac8e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.669982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 768/1559] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.674010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7ed43a7d-fde6-4101-b7e7-1efa40a1a616 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.674134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45ebbc-b667-4283-8fe5-8ec9405839fc None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.674134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45ebbc-b667-4283-8fe5-8ec9405839fc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.674280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7ed43a7d-fde6-4101-b7e7-1efa40a1a616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.674382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45ebbc-b667-4283-8fe5-8ec9405839fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.682437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:08.685142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 792/1560] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.686877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6257eda-ab3d-4d3b-bd42-44438ba5fa4d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.687317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6257eda-ab3d-4d3b-bd42-44438ba5fa4d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.687800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6257eda-ab3d-4d3b-bd42-44438ba5fa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.696034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1a45ebbc-b667-4283-8fe5-8ec9405839fc None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.697448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1a45ebbc-b667-4283-8fe5-8ec9405839fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.707452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 769/1561] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.708882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc84fb-6109-4a19-9a25-fbd4b913f3b0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.709163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc84fb-6109-4a19-9a25-fbd4b913f3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.709480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc84fb-6109-4a19-9a25-fbd4b913f3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.713633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e6257eda-ab3d-4d3b-bd42-44438ba5fa4d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.716002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e6257eda-ab3d-4d3b-bd42-44438ba5fa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.726996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 793/1562] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.728100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6257eda-ab3d-4d3b-bd42-44438ba5fa4d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.730483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0bbc84fb-6109-4a19-9a25-fbd4b913f3b0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.731801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0bbc84fb-6109-4a19-9a25-fbd4b913f3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.740762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 770/1563] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.741196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6257eda-ab3d-4d3b-bd42-44438ba5fa4d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.741615 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc84fb-6109-4a19-9a25-fbd4b913f3b0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.750034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0d67-384a-4fdc-9239-d82ff8fb2e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.750718 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0d67-384a-4fdc-9239-d82ff8fb2e5d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.751003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0d67-384a-4fdc-9239-d82ff8fb2e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.751278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0d67-384a-4fdc-9239-d82ff8fb2e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.751997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac0d67-384a-4fdc-9239-d82ff8fb2e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.754091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac0d67-384a-4fdc-9239-d82ff8fb2e5d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.793229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 794/1564] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:08.793229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0d67-384a-4fdc-9239-d82ff8fb2e5d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.827665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc84fb-6109-4a19-9a25-fbd4b913f3b0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.876977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0d67-384a-4fdc-9239-d82ff8fb2e5d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.882754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7433895-1f94-4a75-bedf-4fd7f17f8e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.883473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7433895-1f94-4a75-bedf-4fd7f17f8e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.883879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7433895-1f94-4a75-bedf-4fd7f17f8e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.884313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7433895-1f94-4a75-bedf-4fd7f17f8e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.885254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7433895-1f94-4a75-bedf-4fd7f17f8e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.887192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7433895-1f94-4a75-bedf-4fd7f17f8e48 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.898598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee1f00-b9b1-4360-8924-024bb9e505bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.900130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee1f00-b9b1-4360-8924-024bb9e505bd service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.900531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee1f00-b9b1-4360-8924-024bb9e505bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.900903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee1f00-b9b1-4360-8924-024bb9e505bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.904852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 795/1565] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:08.906582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cebd0-8212-40f0-9061-6b14fd53f7f3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.906976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cebd0-8212-40f0-9061-6b14fd53f7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.907437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cebd0-8212-40f0-9061-6b14fd53f7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.919383 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee1f00-b9b1-4360-8924-024bb9e505bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.921988 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee1f00-b9b1-4360-8924-024bb9e505bd service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.935637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a97cebd0-8212-40f0-9061-6b14fd53f7f3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.936896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a97cebd0-8212-40f0-9061-6b14fd53f7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.941791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 771/1566] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:08.943343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee1f00-b9b1-4360-8924-024bb9e505bd service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:08.946012 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 796/1567] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.947722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17589a79-a468-4b16-96e3-1ca1938f12a5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.948286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17589a79-a468-4b16-96e3-1ca1938f12a5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.948768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17589a79-a468-4b16-96e3-1ca1938f12a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.961128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee1f00-b9b1-4360-8924-024bb9e505bd service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:08.969647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca651b9-47b3-42df-b4e8-a260840f3858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:08.970830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca651b9-47b3-42df-b4e8-a260840f3858 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.971306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca651b9-47b3-42df-b4e8-a260840f3858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.971768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca651b9-47b3-42df-b4e8-a260840f3858 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:08.972949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-17589a79-a468-4b16-96e3-1ca1938f12a5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:08.973110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca651b9-47b3-42df-b4e8-a260840f3858 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:08.973573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca651b9-47b3-42df-b4e8-a260840f3858 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:08.974579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-17589a79-a468-4b16-96e3-1ca1938f12a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:08.976695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca651b9-47b3-42df-b4e8-a260840f3858 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:08.984476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 797/1568] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:08.985975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886f197-fae5-4428-ae49-38a0a1f1c260 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:08.986310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886f197-fae5-4428-ae49-38a0a1f1c260 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:08.986521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886f197-fae5-4428-ae49-38a0a1f1c260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.001544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 772/1569] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:09.003147 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca651b9-47b3-42df-b4e8-a260840f3858 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.015192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b886f197-fae5-4428-ae49-38a0a1f1c260 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:09.016607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b886f197-fae5-4428-ae49-38a0a1f1c260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:09.022459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca651b9-47b3-42df-b4e8-a260840f3858 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.027309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 798/1570] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:09.028544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886f197-fae5-4428-ae49-38a0a1f1c260 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.032837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a790-4322-4140-a5e8-bc9e233a6dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.034283 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a790-4322-4140-a5e8-bc9e233a6dfa admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.034780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a790-4322-4140-a5e8-bc9e233a6dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.035314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a790-4322-4140-a5e8-bc9e233a6dfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.036779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66a790-4322-4140-a5e8-bc9e233a6dfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:09.037320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66a790-4322-4140-a5e8-bc9e233a6dfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.038990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886f197-fae5-4428-ae49-38a0a1f1c260 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.040840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66a790-4322-4140-a5e8-bc9e233a6dfa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.044527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7d785-6b67-47f4-ada1-5db515c2f78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.045149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7d785-6b67-47f4-ada1-5db515c2f78c admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.045432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7d785-6b67-47f4-ada1-5db515c2f78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.045732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7d785-6b67-47f4-ada1-5db515c2f78c admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.046451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7d785-6b67-47f4-ada1-5db515c2f78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.048469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7d785-6b67-47f4-ada1-5db515c2f78c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.066866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 773/1571] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:09.068434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a790-4322-4140-a5e8-bc9e233a6dfa admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.083615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 799/1572] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:09 2022] POST /identity/v3/users => generated 289 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:09.085056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7d785-6b67-47f4-ada1-5db515c2f78c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.089679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a790-4322-4140-a5e8-bc9e233a6dfa admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.099174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7d785-6b67-47f4-ada1-5db515c2f78c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.099509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7143dd-acdd-46c1-b237-a334bacce8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.100517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7143dd-acdd-46c1-b237-a334bacce8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.101378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7143dd-acdd-46c1-b237-a334bacce8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.101938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7143dd-acdd-46c1-b237-a334bacce8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.103466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7143dd-acdd-46c1-b237-a334bacce8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.106286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa6306-0894-47d4-bb2e-7dd07dce5d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.107661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa6306-0894-47d4-bb2e-7dd07dce5d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.107843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7143dd-acdd-46c1-b237-a334bacce8d8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.108165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa6306-0894-47d4-bb2e-7dd07dce5d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.108647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa6306-0894-47d4-bb2e-7dd07dce5d5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.110343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa6306-0894-47d4-bb2e-7dd07dce5d5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:09.110704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa6306-0894-47d4-bb2e-7dd07dce5d5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.114046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa6306-0894-47d4-bb2e-7dd07dce5d5e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.136732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 774/1573] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:09.136732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7143dd-acdd-46c1-b237-a334bacce8d8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.153971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 800/1574] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:09.153971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa6306-0894-47d4-bb2e-7dd07dce5d5e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.169116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7143dd-acdd-46c1-b237-a334bacce8d8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.175314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa6306-0894-47d4-bb2e-7dd07dce5d5e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.185397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed9cea-9825-47fe-a15f-b67e997908fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.186117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed9cea-9825-47fe-a15f-b67e997908fb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.186919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547a4e6-397c-4e13-8251-55ba65e08094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.188119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547a4e6-397c-4e13-8251-55ba65e08094 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.188587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547a4e6-397c-4e13-8251-55ba65e08094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.189434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547a4e6-397c-4e13-8251-55ba65e08094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.190375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547a4e6-397c-4e13-8251-55ba65e08094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:09.191091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547a4e6-397c-4e13-8251-55ba65e08094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.194576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed9cea-9825-47fe-a15f-b67e997908fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.195755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed9cea-9825-47fe-a15f-b67e997908fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.198093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed9cea-9825-47fe-a15f-b67e997908fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.200653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547a4e6-397c-4e13-8251-55ba65e08094 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.202841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed9cea-9825-47fe-a15f-b67e997908fb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.232866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 775/1575] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:09.242312 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547a4e6-397c-4e13-8251-55ba65e08094 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.267010 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547a4e6-397c-4e13-8251-55ba65e08094 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.279074 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42188143-57cb-4ba9-8958-ec9a78f93b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.281339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42188143-57cb-4ba9-8958-ec9a78f93b7c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.281791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42188143-57cb-4ba9-8958-ec9a78f93b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.282378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42188143-57cb-4ba9-8958-ec9a78f93b7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.284066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42188143-57cb-4ba9-8958-ec9a78f93b7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:09.284686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42188143-57cb-4ba9-8958-ec9a78f93b7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.287984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 801/1576] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:09 2022] POST /identity/v3/projects => generated 389 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:09.289501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42188143-57cb-4ba9-8958-ec9a78f93b7c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.289684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed9cea-9825-47fe-a15f-b67e997908fb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.325240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 776/1577] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:09.325240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42188143-57cb-4ba9-8958-ec9a78f93b7c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.333004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42188143-57cb-4ba9-8958-ec9a78f93b7c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.385002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:09.397599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:09.414951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fb2d4-09b8-4e87-bce9-1d477eca7271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.416017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fb2d4-09b8-4e87-bce9-1d477eca7271 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.416498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fb2d4-09b8-4e87-bce9-1d477eca7271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.417380 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fb2d4-09b8-4e87-bce9-1d477eca7271 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.418199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fb2d4-09b8-4e87-bce9-1d477eca7271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.420087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fb2d4-09b8-4e87-bce9-1d477eca7271 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.427895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed9cea-9825-47fe-a15f-b67e997908fb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.436945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff86ee2-7da5-4365-a454-93843815b927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.437779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 777/1578] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:09.438578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff86ee2-7da5-4365-a454-93843815b927 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.438859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fb2d4-09b8-4e87-bce9-1d477eca7271 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.439058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff86ee2-7da5-4365-a454-93843815b927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.439599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff86ee2-7da5-4365-a454-93843815b927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.440914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff86ee2-7da5-4365-a454-93843815b927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.444809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff86ee2-7da5-4365-a454-93843815b927 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.452303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fb2d4-09b8-4e87-bce9-1d477eca7271 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.457630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798a5e9-d3f1-4cc7-9e69-2ec4dbfa8770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.458594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798a5e9-d3f1-4cc7-9e69-2ec4dbfa8770 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.459037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798a5e9-d3f1-4cc7-9e69-2ec4dbfa8770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.459349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798a5e9-d3f1-4cc7-9e69-2ec4dbfa8770 admin admin] PATH_INFO: `/v3/projects/42d8a0a3a9204fae93eb16754608827b/users/4e354954d5264bd199b0111f9f41efa2/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.493580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798a5e9-d3f1-4cc7-9e69-2ec4dbfa8770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d8a0a3a9204fae93eb16754608827b, user_id=4e354954d5264bd199b0111f9f41efa2, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.497820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798a5e9-d3f1-4cc7-9e69-2ec4dbfa8770 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.505898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 802/1579] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:09 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:09.505898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff86ee2-7da5-4365-a454-93843815b927 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.527175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 778/1580] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:09 2022] PUT /identity/v3/projects/42d8a0a3a9204fae93eb16754608827b/users/4e354954d5264bd199b0111f9f41efa2/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:09.529392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798a5e9-d3f1-4cc7-9e69-2ec4dbfa8770 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.647972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798a5e9-d3f1-4cc7-9e69-2ec4dbfa8770 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.655164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbcef0-d0bb-4f42-9a26-a9d1b162cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.655955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbcef0-d0bb-4f42-9a26-a9d1b162cee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.656593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbcef0-d0bb-4f42-9a26-a9d1b162cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.657130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbcef0-d0bb-4f42-9a26-a9d1b162cee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.658052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbcef0-d0bb-4f42-9a26-a9d1b162cee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.660481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbcef0-d0bb-4f42-9a26-a9d1b162cee8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.667502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff86ee2-7da5-4365-a454-93843815b927 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.674368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbbf0c-0391-4709-a196-fd5cb782270b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.675348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbbf0c-0391-4709-a196-fd5cb782270b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.676666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbbf0c-0391-4709-a196-fd5cb782270b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.677429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbbf0c-0391-4709-a196-fd5cb782270b admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.678337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbbf0c-0391-4709-a196-fd5cb782270b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.681759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbbf0c-0391-4709-a196-fd5cb782270b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.707167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 779/1581] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:09 2022] POST /identity/v3/projects => generated 403 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:09.707167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbcef0-d0bb-4f42-9a26-a9d1b162cee8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.732893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 803/1582] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:09 2022] POST /identity/v3/projects => generated 375 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:09.732893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbbf0c-0391-4709-a196-fd5cb782270b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.813599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbcef0-d0bb-4f42-9a26-a9d1b162cee8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.842062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbbf0c-0391-4709-a196-fd5cb782270b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.850707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81f138-cc76-431f-88ee-69107639983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.851887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81f138-cc76-431f-88ee-69107639983b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.852315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81f138-cc76-431f-88ee-69107639983b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.852739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81f138-cc76-431f-88ee-69107639983b admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.853939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81f138-cc76-431f-88ee-69107639983b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.858269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81f138-cc76-431f-88ee-69107639983b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.879386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:09.894094 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b0c88-acd7-4034-8ca0-5dc8a3d7d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:09.894996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b0c88-acd7-4034-8ca0-5dc8a3d7d3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:09.895357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b0c88-acd7-4034-8ca0-5dc8a3d7d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:09.895796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b0c88-acd7-4034-8ca0-5dc8a3d7d3d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:09.896574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b0c88-acd7-4034-8ca0-5dc8a3d7d3d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:09.897874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-514b0c88-acd7-4034-8ca0-5dc8a3d7d3d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:22:09.897874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jun 06 14:22:09.897874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: Jun 06 14:22:09.899164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b0c88-acd7-4034-8ca0-5dc8a3d7d3d4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:09.901654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 804/1583] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:09 2022] POST /identity/v3/projects => generated 393 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:09.906751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81f138-cc76-431f-88ee-69107639983b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.926822 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 780/1584] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:09 2022] POST /identity/v3/users => generated 295 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:09.930925 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b0c88-acd7-4034-8ca0-5dc8a3d7d3d4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:09.945136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b0c88-acd7-4034-8ca0-5dc8a3d7d3d4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:09.980026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:10.019400 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e67d6-895a-4d0c-b349-46f34c1603c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.020279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e67d6-895a-4d0c-b349-46f34c1603c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.020406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e67d6-895a-4d0c-b349-46f34c1603c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.020626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e67d6-895a-4d0c-b349-46f34c1603c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.021336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e67d6-895a-4d0c-b349-46f34c1603c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.023376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e67d6-895a-4d0c-b349-46f34c1603c0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.028211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81f138-cc76-431f-88ee-69107639983b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.036398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b841ccd-f63c-4fc7-a240-c56870170f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.038791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b841ccd-f63c-4fc7-a240-c56870170f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.039407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b841ccd-f63c-4fc7-a240-c56870170f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.039979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b841ccd-f63c-4fc7-a240-c56870170f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.041524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b841ccd-f63c-4fc7-a240-c56870170f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.046134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b841ccd-f63c-4fc7-a240-c56870170f7a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.051658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 781/1585] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:09 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:10.051658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedd5de-3572-421e-9810-8b4990531c04 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.051658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedd5de-3572-421e-9810-8b4990531c04 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.051658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedd5de-3572-421e-9810-8b4990531c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.070006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 805/1586] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.071295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b841ccd-f63c-4fc7-a240-c56870170f7a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.092383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b841ccd-f63c-4fc7-a240-c56870170f7a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.099245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8fedd5de-3572-421e-9810-8b4990531c04 None None] MFA Rules not processed for user `4e354954d5264bd199b0111f9f41efa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:10.100748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6728-c6dc-4486-8e08-65a0e6ba1301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.101384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8fedd5de-3572-421e-9810-8b4990531c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:10.101658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6728-c6dc-4486-8e08-65a0e6ba1301 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.102008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6728-c6dc-4486-8e08-65a0e6ba1301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.102323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6728-c6dc-4486-8e08-65a0e6ba1301 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.103195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae6728-c6dc-4486-8e08-65a0e6ba1301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.107183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae6728-c6dc-4486-8e08-65a0e6ba1301 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.138993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 806/1587] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:10 2022] POST /identity/v3/users => generated 302 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:10.144255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6728-c6dc-4486-8e08-65a0e6ba1301 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.165138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6728-c6dc-4486-8e08-65a0e6ba1301 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.176433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0df02-ced6-466a-8cf9-958738f86ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.178211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0df02-ced6-466a-8cf9-958738f86ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.179063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0df02-ced6-466a-8cf9-958738f86ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.179729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0df02-ced6-466a-8cf9-958738f86ba9 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.181149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0df02-ced6-466a-8cf9-958738f86ba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.185400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0df02-ced6-466a-8cf9-958738f86ba9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.204004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 782/1588] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:10.207560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedd5de-3572-421e-9810-8b4990531c04 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.228050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 807/1589] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:10 2022] POST /identity/v3/users => generated 288 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:10.228440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedd5de-3572-421e-9810-8b4990531c04 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.230380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0df02-ced6-466a-8cf9-958738f86ba9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.241251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decad30-b81f-4b05-a0df-a156386fe5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.242523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decad30-b81f-4b05-a0df-a156386fe5de admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.243019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decad30-b81f-4b05-a0df-a156386fe5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.243549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decad30-b81f-4b05-a0df-a156386fe5de admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.244029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0df02-ced6-466a-8cf9-958738f86ba9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.244908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6decad30-b81f-4b05-a0df-a156386fe5de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.249085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6decad30-b81f-4b05-a0df-a156386fe5de admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.250285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a64a6-5aeb-43b2-9bf8-f9b7e593fa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.251064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a64a6-5aeb-43b2-9bf8-f9b7e593fa4f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.251529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a64a6-5aeb-43b2-9bf8-f9b7e593fa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.251987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a64a6-5aeb-43b2-9bf8-f9b7e593fa4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.252753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a64a6-5aeb-43b2-9bf8-f9b7e593fa4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.254515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a64a6-5aeb-43b2-9bf8-f9b7e593fa4f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.269916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 808/1590] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.270976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a64a6-5aeb-43b2-9bf8-f9b7e593fa4f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.282795 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a64a6-5aeb-43b2-9bf8-f9b7e593fa4f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.285459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 783/1591] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:10 2022] POST /identity/v3/users => generated 297 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:10.287463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decad30-b81f-4b05-a0df-a156386fe5de admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.287917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d39b8-846a-40dc-867e-ad63e63cd495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.288610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d39b8-846a-40dc-867e-ad63e63cd495 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.289295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d39b8-846a-40dc-867e-ad63e63cd495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.289576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d39b8-846a-40dc-867e-ad63e63cd495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.290370 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d39b8-846a-40dc-867e-ad63e63cd495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.293350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d39b8-846a-40dc-867e-ad63e63cd495 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.302751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decad30-b81f-4b05-a0df-a156386fe5de admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.308887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf9000-dfd2-4019-bf25-d3f836166cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.309482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf9000-dfd2-4019-bf25-d3f836166cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.309781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf9000-dfd2-4019-bf25-d3f836166cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.310065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf9000-dfd2-4019-bf25-d3f836166cf8 admin admin] PATH_INFO: `/v3/projects/c2ab6d13d539433a853fc6805d819b85/users/a5fee90ee6694ae4a821f60ca194ce84/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.315238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 809/1592] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.317301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d39b8-846a-40dc-867e-ad63e63cd495 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.332644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf9000-dfd2-4019-bf25-d3f836166cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ab6d13d539433a853fc6805d819b85, user_id=a5fee90ee6694ae4a821f60ca194ce84, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.335164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d39b8-846a-40dc-867e-ad63e63cd495 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.335543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf9000-dfd2-4019-bf25-d3f836166cf8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.342376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f18113-05ea-4035-9c44-f5d604ad9962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.343403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f18113-05ea-4035-9c44-f5d604ad9962 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.343663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f18113-05ea-4035-9c44-f5d604ad9962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.343993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f18113-05ea-4035-9c44-f5d604ad9962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.344974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f18113-05ea-4035-9c44-f5d604ad9962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.347134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f18113-05ea-4035-9c44-f5d604ad9962 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.364713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 784/1593] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:10 2022] PUT /identity/v3/projects/c2ab6d13d539433a853fc6805d819b85/users/a5fee90ee6694ae4a821f60ca194ce84/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:10.366872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf9000-dfd2-4019-bf25-d3f836166cf8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.371411 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 810/1594] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.372904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f18113-05ea-4035-9c44-f5d604ad9962 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.387387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f18113-05ea-4035-9c44-f5d604ad9962 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.416092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:10.487998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf9000-dfd2-4019-bf25-d3f836166cf8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.497657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daccb46-722a-4a56-8268-f9f05f0e3c53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.500551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daccb46-722a-4a56-8268-f9f05f0e3c53 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.502101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daccb46-722a-4a56-8268-f9f05f0e3c53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.503694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daccb46-722a-4a56-8268-f9f05f0e3c53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.538457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc870e-d504-4b2f-910d-96c5f0252c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.539871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc870e-d504-4b2f-910d-96c5f0252c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.540389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc870e-d504-4b2f-910d-96c5f0252c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.540932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc870e-d504-4b2f-910d-96c5f0252c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.542291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc870e-d504-4b2f-910d-96c5f0252c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.545806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc870e-d504-4b2f-910d-96c5f0252c1d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.576142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 811/1595] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.577178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc870e-d504-4b2f-910d-96c5f0252c1d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.605410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc870e-d504-4b2f-910d-96c5f0252c1d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.611768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206fe9-2238-4a2d-b890-91897c2fedc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.612797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206fe9-2238-4a2d-b890-91897c2fedc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.613303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206fe9-2238-4a2d-b890-91897c2fedc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.613788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206fe9-2238-4a2d-b890-91897c2fedc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.614752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6206fe9-2238-4a2d-b890-91897c2fedc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.616653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6206fe9-2238-4a2d-b890-91897c2fedc9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.632860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 812/1596] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.634142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206fe9-2238-4a2d-b890-91897c2fedc9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.647199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206fe9-2238-4a2d-b890-91897c2fedc9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.649611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daccb46-722a-4a56-8268-f9f05f0e3c53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.654577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618ee2f-1ec3-42db-a851-1cf1dacc6aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.655553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618ee2f-1ec3-42db-a851-1cf1dacc6aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.655994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618ee2f-1ec3-42db-a851-1cf1dacc6aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.656460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618ee2f-1ec3-42db-a851-1cf1dacc6aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.656937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daccb46-722a-4a56-8268-f9f05f0e3c53 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.657570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618ee2f-1ec3-42db-a851-1cf1dacc6aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.660342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618ee2f-1ec3-42db-a851-1cf1dacc6aa6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.673275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 813/1597] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.674310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618ee2f-1ec3-42db-a851-1cf1dacc6aa6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.686704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 785/1598] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:10.688639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634357d-4d23-4072-88d3-5e7967fd91f0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.689293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634357d-4d23-4072-88d3-5e7967fd91f0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.689592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634357d-4d23-4072-88d3-5e7967fd91f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.700810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618ee2f-1ec3-42db-a851-1cf1dacc6aa6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.707065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b7593-7146-4ed8-a3a5-0972bef58611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.707938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b7593-7146-4ed8-a3a5-0972bef58611 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.708223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b7593-7146-4ed8-a3a5-0972bef58611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.708661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b7593-7146-4ed8-a3a5-0972bef58611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.709724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b7593-7146-4ed8-a3a5-0972bef58611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.711472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b7593-7146-4ed8-a3a5-0972bef58611 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.730456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 814/1599] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.731749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b7593-7146-4ed8-a3a5-0972bef58611 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.734639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f634357d-4d23-4072-88d3-5e7967fd91f0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:10.736951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f634357d-4d23-4072-88d3-5e7967fd91f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:10.755663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 786/1600] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:10.755663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634357d-4d23-4072-88d3-5e7967fd91f0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.765352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634357d-4d23-4072-88d3-5e7967fd91f0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.771031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f1b74-e6c5-49f8-a868-6bfdd6717b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.771728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f1b74-e6c5-49f8-a868-6bfdd6717b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.772083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f1b74-e6c5-49f8-a868-6bfdd6717b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.772346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f1b74-e6c5-49f8-a868-6bfdd6717b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.773185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f1b74-e6c5-49f8-a868-6bfdd6717b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.773958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b7593-7146-4ed8-a3a5-0972bef58611 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.775129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f1b74-e6c5-49f8-a868-6bfdd6717b0d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.791520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 787/1601] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.793650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91164-3a84-41b4-aa3b-1069705d9441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.793930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f1b74-e6c5-49f8-a868-6bfdd6717b0d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.794612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91164-3a84-41b4-aa3b-1069705d9441 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.794912 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91164-3a84-41b4-aa3b-1069705d9441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.795368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91164-3a84-41b4-aa3b-1069705d9441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.796876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c91164-3a84-41b4-aa3b-1069705d9441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.800264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c91164-3a84-41b4-aa3b-1069705d9441 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.804956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f1b74-e6c5-49f8-a868-6bfdd6717b0d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.811255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02d785-3eb8-445c-b6f5-a0923b4f3900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:10.812232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02d785-3eb8-445c-b6f5-a0923b4f3900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:10.812520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02d785-3eb8-445c-b6f5-a0923b4f3900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:10.812792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02d785-3eb8-445c-b6f5-a0923b4f3900 admin admin] PATH_INFO: `/v3/projects/52da965db3654006abddd5b40b0905c2/users/405ac2f6c07545efaa818e10fc788c05/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:10.837339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02d785-3eb8-445c-b6f5-a0923b4f3900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52da965db3654006abddd5b40b0905c2, user_id=405ac2f6c07545efaa818e10fc788c05, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:10.840931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02d785-3eb8-445c-b6f5-a0923b4f3900 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:10.845444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 815/1602] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:10.847218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91164-3a84-41b4-aa3b-1069705d9441 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.860857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 788/1603] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:10 2022] PUT /identity/v3/projects/52da965db3654006abddd5b40b0905c2/users/405ac2f6c07545efaa818e10fc788c05/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:10.861565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02d785-3eb8-445c-b6f5-a0923b4f3900 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:10.901654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91164-3a84-41b4-aa3b-1069705d9441 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:10.934010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:11.014555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02d785-3eb8-445c-b6f5-a0923b4f3900 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:11.026093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963814e-2b3d-44c6-b7c8-ec5a4ad35ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:11.027791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963814e-2b3d-44c6-b7c8-ec5a4ad35ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:11.028073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963814e-2b3d-44c6-b7c8-ec5a4ad35ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:11.028346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963814e-2b3d-44c6-b7c8-ec5a4ad35ea7 admin admin] PATH_INFO: `/v3/projects/7f970147e5094c01bacf052bbc47c129/users/7c73fcf874e2461e920859b5c3ee6599/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:11.044695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c48a85-d545-4ff7-948e-7082523b63c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:11.045401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c48a85-d545-4ff7-948e-7082523b63c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:11.045685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c48a85-d545-4ff7-948e-7082523b63c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:11.045981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c48a85-d545-4ff7-948e-7082523b63c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:11.046754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c48a85-d545-4ff7-948e-7082523b63c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:11.048408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c48a85-d545-4ff7-948e-7082523b63c3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:11.068112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 816/1604] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:11.070012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963814e-2b3d-44c6-b7c8-ec5a4ad35ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f970147e5094c01bacf052bbc47c129, user_id=7c73fcf874e2461e920859b5c3ee6599, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:11.074792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963814e-2b3d-44c6-b7c8-ec5a4ad35ea7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:11.076115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c48a85-d545-4ff7-948e-7082523b63c3 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:11.111904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 789/1605] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:10 2022] PUT /identity/v3/projects/7f970147e5094c01bacf052bbc47c129/users/7c73fcf874e2461e920859b5c3ee6599/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:11.112890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963814e-2b3d-44c6-b7c8-ec5a4ad35ea7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:11.133512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:11.238477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c48a85-d545-4ff7-948e-7082523b63c3 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:11.260670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963814e-2b3d-44c6-b7c8-ec5a4ad35ea7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:11.271683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac4771-c19e-48e5-8bd3-86d83b662725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:11.273310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac4771-c19e-48e5-8bd3-86d83b662725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:11.273876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac4771-c19e-48e5-8bd3-86d83b662725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:11.274501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac4771-c19e-48e5-8bd3-86d83b662725 admin admin] PATH_INFO: `/v3/projects/c41996fbf4ab4b81bc47ac29252ec762/users/3f2e1c5d022c4bde9e76b8a933739557/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:11.319059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac4771-c19e-48e5-8bd3-86d83b662725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41996fbf4ab4b81bc47ac29252ec762, user_id=3f2e1c5d022c4bde9e76b8a933739557, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:11.326025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac4771-c19e-48e5-8bd3-86d83b662725 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:11.357021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 790/1606] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:11 2022] PUT /identity/v3/projects/c41996fbf4ab4b81bc47ac29252ec762/users/3f2e1c5d022c4bde9e76b8a933739557/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:11.358733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac4771-c19e-48e5-8bd3-86d83b662725 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:11.381494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273bc84-51ee-495f-95be-2d9a129a69a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:11.382255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273bc84-51ee-495f-95be-2d9a129a69a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:11.382592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273bc84-51ee-495f-95be-2d9a129a69a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:11.383090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273bc84-51ee-495f-95be-2d9a129a69a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:11.394770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273bc84-51ee-495f-95be-2d9a129a69a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:11.398926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273bc84-51ee-495f-95be-2d9a129a69a6 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:11.412003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 817/1607] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:11.413584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc38c8-cd17-4f73-8d6e-d6224fc9dd59 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:11.414048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc38c8-cd17-4f73-8d6e-d6224fc9dd59 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:11.414342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc38c8-cd17-4f73-8d6e-d6224fc9dd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:11.435726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f7dc38c8-cd17-4f73-8d6e-d6224fc9dd59 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:11.437003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f7dc38c8-cd17-4f73-8d6e-d6224fc9dd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:11.528582 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac4771-c19e-48e5-8bd3-86d83b662725 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:11.538206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f802a-f4e5-4869-8afb-9d83bc61e7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:11.540395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f802a-f4e5-4869-8afb-9d83bc61e7a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:11.541143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f802a-f4e5-4869-8afb-9d83bc61e7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:11.542921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f802a-f4e5-4869-8afb-9d83bc61e7a1 admin admin] PATH_INFO: `/v3/projects/001d77c3bc65465dbaa77d48cdb08b0a/users/c41c1f9bd20c4e52a472a042f156ae3c/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:11.545403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 818/1608] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:11.552098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc38c8-cd17-4f73-8d6e-d6224fc9dd59 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:11.586550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc38c8-cd17-4f73-8d6e-d6224fc9dd59 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:11.595491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafa1b3-0c22-48b1-a7f5-b775f124676d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:11.599558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa1b3-0c22-48b1-a7f5-b775f124676d admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:11.599979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa1b3-0c22-48b1-a7f5-b775f124676d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:11.600678 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa1b3-0c22-48b1-a7f5-b775f124676d admin admin] PATH_INFO: `/v3/projects/5e2e414f37bf4a89ac18e1836c793b0b/users/86f9bcac527c45d6bc81b70e26544924/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:11.629021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f802a-f4e5-4869-8afb-9d83bc61e7a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001d77c3bc65465dbaa77d48cdb08b0a, user_id=c41c1f9bd20c4e52a472a042f156ae3c, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:11.635307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f802a-f4e5-4869-8afb-9d83bc61e7a1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:11.640563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:11.653934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafa1b3-0c22-48b1-a7f5-b775f124676d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e2e414f37bf4a89ac18e1836c793b0b, user_id=86f9bcac527c45d6bc81b70e26544924, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:11.655026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-ccafa1b3-0c22-48b1-a7f5-b775f124676d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:22:11.655026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: warnings.warn(msg) Jun 06 14:22:11.655026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: Jun 06 14:22:11.657180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafa1b3-0c22-48b1-a7f5-b775f124676d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:11.674367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 791/1609] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:11 2022] PUT /identity/v3/projects/001d77c3bc65465dbaa77d48cdb08b0a/users/c41c1f9bd20c4e52a472a042f156ae3c/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:11.674367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588864e4-75e8-4700-8e4f-15b7c3e673d0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:11.674367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588864e4-75e8-4700-8e4f-15b7c3e673d0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:11.674367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588864e4-75e8-4700-8e4f-15b7c3e673d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:11.700007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 819/1610] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:11 2022] PUT /identity/v3/projects/5e2e414f37bf4a89ac18e1836c793b0b/users/86f9bcac527c45d6bc81b70e26544924/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:11.700007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef080b-7719-4ee9-af1c-63609858903a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:11.700007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef080b-7719-4ee9-af1c-63609858903a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:11.700007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef080b-7719-4ee9-af1c-63609858903a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:11.704413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-588864e4-75e8-4700-8e4f-15b7c3e673d0 None None] MFA Rules not processed for user `7c73fcf874e2461e920859b5c3ee6599`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:11.706571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-588864e4-75e8-4700-8e4f-15b7c3e673d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:11.743154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-79ef080b-7719-4ee9-af1c-63609858903a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:11.746447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-79ef080b-7719-4ee9-af1c-63609858903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:11.833428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 792/1611] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:11.834328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588864e4-75e8-4700-8e4f-15b7c3e673d0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:11.880600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:12.010299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 820/1612] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:12.014114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44de2b3-87f4-48a0-aac0-95ea069545e9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.014405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44de2b3-87f4-48a0-aac0-95ea069545e9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.014686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44de2b3-87f4-48a0-aac0-95ea069545e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.032423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588864e4-75e8-4700-8e4f-15b7c3e673d0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:12.044906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b87f2-0226-4152-a6d7-0aab01c3259b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:12.045673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b87f2-0226-4152-a6d7-0aab01c3259b service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.045906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b87f2-0226-4152-a6d7-0aab01c3259b service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.046223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b87f2-0226-4152-a6d7-0aab01c3259b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.068445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b87f2-0226-4152-a6d7-0aab01c3259b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:12.069979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b87f2-0226-4152-a6d7-0aab01c3259b service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:12.089662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 793/1613] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:12.091110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6d4f-874f-4d6d-aa92-c7aa381c5023 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.091407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6d4f-874f-4d6d-aa92-c7aa381c5023 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.091733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6d4f-874f-4d6d-aa92-c7aa381c5023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.099944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d44de2b3-87f4-48a0-aac0-95ea069545e9 None None] MFA Rules not processed for user `c41c1f9bd20c4e52a472a042f156ae3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:12.106891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d44de2b3-87f4-48a0-aac0-95ea069545e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:12.137890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a6fd6d4f-874f-4d6d-aa92-c7aa381c5023 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:12.139129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a6fd6d4f-874f-4d6d-aa92-c7aa381c5023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:12.169677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 794/1614] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:12.171155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4fe4c-6e4e-4945-a805-fb45f794c316 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.171440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4fe4c-6e4e-4945-a805-fb45f794c316 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.171716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4fe4c-6e4e-4945-a805-fb45f794c316 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.173430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 795/1615] 10.222.0.13 () {54 vars in 961 bytes} [Mon Jun 6 14:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:22:12.174284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4fe4c-6e4e-4945-a805-fb45f794c316 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:12.212749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4fe4c-6e4e-4945-a805-fb45f794c316 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:12.221443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a98475-1841-4c5d-b365-a2ad3c7534b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:12.222256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a98475-1841-4c5d-b365-a2ad3c7534b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.222552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a98475-1841-4c5d-b365-a2ad3c7534b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.225864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a98475-1841-4c5d-b365-a2ad3c7534b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.238792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a98475-1841-4c5d-b365-a2ad3c7534b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:12.241098 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a98475-1841-4c5d-b365-a2ad3c7534b7 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:12.251194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 821/1616] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:12.251194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4582b-1015-4baf-8455-c4ce3396a7ff None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.251194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4582b-1015-4baf-8455-c4ce3396a7ff None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.251194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4582b-1015-4baf-8455-c4ce3396a7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.264157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 796/1617] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:12.267706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a98475-1841-4c5d-b365-a2ad3c7534b7 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:12.292353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a98475-1841-4c5d-b365-a2ad3c7534b7 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:12.309449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b0d4582b-1015-4baf-8455-c4ce3396a7ff None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:12.317974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b0d4582b-1015-4baf-8455-c4ce3396a7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:12.324482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d83ac-307a-47f4-b74f-0b02011ae900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:12.325280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d83ac-307a-47f4-b74f-0b02011ae900 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.325565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d83ac-307a-47f4-b74f-0b02011ae900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.325857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d83ac-307a-47f4-b74f-0b02011ae900 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.330689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d83ac-307a-47f4-b74f-0b02011ae900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:12.333069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d83ac-307a-47f4-b74f-0b02011ae900 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:12.408683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 797/1618] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:12 2022] POST /identity/v3/projects => generated 403 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:12.410667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d83ac-307a-47f4-b74f-0b02011ae900 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:12.529666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 822/1619] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Jun 6 14:22:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:12.531335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4582b-1015-4baf-8455-c4ce3396a7ff None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:12.572608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d83ac-307a-47f4-b74f-0b02011ae900 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:12.578349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4582b-1015-4baf-8455-c4ce3396a7ff None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:12.597271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6e864-1f52-4d43-a5fe-24369141b0bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:12.598000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6e864-1f52-4d43-a5fe-24369141b0bd service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.599018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6e864-1f52-4d43-a5fe-24369141b0bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.599321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6e864-1f52-4d43-a5fe-24369141b0bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.615038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:12.735219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6e864-1f52-4d43-a5fe-24369141b0bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:12.738461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6e864-1f52-4d43-a5fe-24369141b0bd service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:12.771653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 798/1620] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:12.772942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6e864-1f52-4d43-a5fe-24369141b0bd service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:12.774408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb840-6ba9-4389-a241-b428d2cf7d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:12.776934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb840-6ba9-4389-a241-b428d2cf7d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.777610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb840-6ba9-4389-a241-b428d2cf7d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.778256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb840-6ba9-4389-a241-b428d2cf7d5e admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.780283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb840-6ba9-4389-a241-b428d2cf7d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:12.798875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb840-6ba9-4389-a241-b428d2cf7d5e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:12.847653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:12.888202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 823/1621] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:12 2022] POST /identity/v3/users => generated 308 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:12.891365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb840-6ba9-4389-a241-b428d2cf7d5e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:12.925475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb840-6ba9-4389-a241-b428d2cf7d5e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:12.941354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101640b7-2fc0-4f97-b1a6-808facf234c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:12.941943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101640b7-2fc0-4f97-b1a6-808facf234c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.942287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101640b7-2fc0-4f97-b1a6-808facf234c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.942587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101640b7-2fc0-4f97-b1a6-808facf234c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:12.944317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101640b7-2fc0-4f97-b1a6-808facf234c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:12.947745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101640b7-2fc0-4f97-b1a6-808facf234c9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:12.969812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 824/1622] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:12.972262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101640b7-2fc0-4f97-b1a6-808facf234c9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:12.976352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6e864-1f52-4d43-a5fe-24369141b0bd service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:12.984319 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33d0db-13f5-4378-a73f-136387983692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:12.985073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33d0db-13f5-4378-a73f-136387983692 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:12.985412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33d0db-13f5-4378-a73f-136387983692 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:12.985843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33d0db-13f5-4378-a73f-136387983692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:13.018737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101640b7-2fc0-4f97-b1a6-808facf234c9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:13.040451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9c770-b123-4efe-8829-ce59d1b917ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:13.044066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9c770-b123-4efe-8829-ce59d1b917ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:13.045452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9c770-b123-4efe-8829-ce59d1b917ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:13.046040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9c770-b123-4efe-8829-ce59d1b917ee admin admin] PATH_INFO: `/v3/projects/3ea67b48983d49b8a5a12c85e5d28715/users/3d1cef9ba36046ca82d716796098954c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:13.091488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33d0db-13f5-4378-a73f-136387983692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:13.094238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33d0db-13f5-4378-a73f-136387983692 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:13.108281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 799/1623] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:13.115815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9c770-b123-4efe-8829-ce59d1b917ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea67b48983d49b8a5a12c85e5d28715, user_id=3d1cef9ba36046ca82d716796098954c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:13.124958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9c770-b123-4efe-8829-ce59d1b917ee admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:13.171082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 825/1624] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:12 2022] PUT /identity/v3/projects/3ea67b48983d49b8a5a12c85e5d28715/users/3d1cef9ba36046ca82d716796098954c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:13.178160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33d0db-13f5-4378-a73f-136387983692 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:13.241427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33d0db-13f5-4378-a73f-136387983692 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:13.499328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeac4b9-d7b2-4c24-a80f-250897ee6df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:13.500370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeac4b9-d7b2-4c24-a80f-250897ee6df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:13.500876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeac4b9-d7b2-4c24-a80f-250897ee6df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:13.501372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeac4b9-d7b2-4c24-a80f-250897ee6df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:13.502606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeac4b9-d7b2-4c24-a80f-250897ee6df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:13.505481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeac4b9-d7b2-4c24-a80f-250897ee6df9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:13.556302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 800/1625] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:22:13.558666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9c770-b123-4efe-8829-ce59d1b917ee admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:13.609516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9c770-b123-4efe-8829-ce59d1b917ee admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:13.647576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcd36a-496b-4d33-80b9-80677eddb24e None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:13.648042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcd36a-496b-4d33-80b9-80677eddb24e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:13.648491 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcd36a-496b-4d33-80b9-80677eddb24e None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:13.648614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748d80f-addb-4061-81d7-34d0951571fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:13.651495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 801/1626] 10.222.0.13 () {54 vars in 912 bytes} [Mon Jun 6 14:22:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:22:13.656820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748d80f-addb-4061-81d7-34d0951571fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:13.657356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748d80f-addb-4061-81d7-34d0951571fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:13.660758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748d80f-addb-4061-81d7-34d0951571fd admin admin] PATH_INFO: `/v3/projects/3ea67b48983d49b8a5a12c85e5d28715/users/3d1cef9ba36046ca82d716796098954c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:13.662726 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9b4e3-dd7f-4740-955a-048a22f97780 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:13.663249 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9b4e3-dd7f-4740-955a-048a22f97780 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:13.663906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9b4e3-dd7f-4740-955a-048a22f97780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:13.680510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748d80f-addb-4061-81d7-34d0951571fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea67b48983d49b8a5a12c85e5d28715, user_id=3d1cef9ba36046ca82d716796098954c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:13.708208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748d80f-addb-4061-81d7-34d0951571fd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:13.726076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fdf9b4e3-dd7f-4740-955a-048a22f97780 None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:13.727348 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fdf9b4e3-dd7f-4740-955a-048a22f97780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:13.784197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 826/1627] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:13 2022] PUT /identity/v3/projects/3ea67b48983d49b8a5a12c85e5d28715/users/3d1cef9ba36046ca82d716796098954c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:13.790473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa8b73-0a1c-4953-abfc-8ef69f376b93 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:13.790823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa8b73-0a1c-4953-abfc-8ef69f376b93 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:13.791047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa8b73-0a1c-4953-abfc-8ef69f376b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:13.852413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:13.873326 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-91fa8b73-0a1c-4953-abfc-8ef69f376b93 None None] MFA Rules not processed for user `3d1cef9ba36046ca82d716796098954c`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:13.876695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-91fa8b73-0a1c-4953-abfc-8ef69f376b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:13.908977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 802/1628] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:22:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:13.927141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9b4e3-dd7f-4740-955a-048a22f97780 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:14.052703 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 827/1629] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:14.066028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9b4e3-dd7f-4740-955a-048a22f97780 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:14.076495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa8b73-0a1c-4953-abfc-8ef69f376b93 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:14.077214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9364f0-c77e-451a-996f-b2c6e457fd2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:14.078047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9364f0-c77e-451a-996f-b2c6e457fd2f service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:14.078439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9364f0-c77e-451a-996f-b2c6e457fd2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:14.078885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9364f0-c77e-451a-996f-b2c6e457fd2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:14.094548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa8b73-0a1c-4953-abfc-8ef69f376b93 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:14.103906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9364f0-c77e-451a-996f-b2c6e457fd2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:14.106973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9364f0-c77e-451a-996f-b2c6e457fd2f service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:14.108084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e5788-a7da-4046-aed9-ac03b504f937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:14.108955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e5788-a7da-4046-aed9-ac03b504f937 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:14.109446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e5788-a7da-4046-aed9-ac03b504f937 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:14.110050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e5788-a7da-4046-aed9-ac03b504f937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:14.134809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e5788-a7da-4046-aed9-ac03b504f937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:14.138147 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 803/1630] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:22:14.140403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e5788-a7da-4046-aed9-ac03b504f937 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:14.161134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 828/1631] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:14 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:22:14.460312 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b033b-cd54-4b49-befe-c2fc8ff73788 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:14.460585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b033b-cd54-4b49-befe-c2fc8ff73788 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:14.461026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b033b-cd54-4b49-befe-c2fc8ff73788 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:14.463180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 804/1632] 10.222.0.13 () {54 vars in 961 bytes} [Mon Jun 6 14:22:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 14:22:14.543064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff81b5-8f1b-4bc2-b919-26b0df29e049 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:14.543862 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff81b5-8f1b-4bc2-b919-26b0df29e049 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:14.544393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff81b5-8f1b-4bc2-b919-26b0df29e049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:14.589107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a7ff81b5-8f1b-4bc2-b919-26b0df29e049 None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:14.601361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a7ff81b5-8f1b-4bc2-b919-26b0df29e049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:14.628935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b033b-cd54-4b49-befe-c2fc8ff73788 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:14.642971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 829/1633] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Jun 6 14:22:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:14.653723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2c3a8-2c84-47f7-95d2-a57b28310da8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:14.654198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2c3a8-2c84-47f7-95d2-a57b28310da8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:14.654586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2c3a8-2c84-47f7-95d2-a57b28310da8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:14.656668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 830/1634] 10.222.0.13 () {54 vars in 912 bytes} [Mon Jun 6 14:22:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:22:14.665510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779435f0-ea68-41a0-885c-211ddda58153 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:14.666006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779435f0-ea68-41a0-885c-211ddda58153 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:14.666545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779435f0-ea68-41a0-885c-211ddda58153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:14.719886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-779435f0-ea68-41a0-885c-211ddda58153 None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:14.726031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-779435f0-ea68-41a0-885c-211ddda58153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:14.756947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 831/1635] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:22:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:14.776526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779435f0-ea68-41a0-885c-211ddda58153 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:14.791955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779435f0-ea68-41a0-885c-211ddda58153 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:14.803433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627aeb0-c771-4da2-9001-706082f636ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:14.804291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627aeb0-c771-4da2-9001-706082f636ea service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:14.804696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627aeb0-c771-4da2-9001-706082f636ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:14.806960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627aeb0-c771-4da2-9001-706082f636ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:14.821457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b033b-cd54-4b49-befe-c2fc8ff73788 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:14.834088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:14.835499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:14.836605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627aeb0-c771-4da2-9001-706082f636ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:14.837040 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:14.839716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] PATH_INFO: `/v3/users/c41c1f9bd20c4e52a472a042f156ae3c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:14.841340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627aeb0-c771-4da2-9001-706082f636ea service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:14.842308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41c1f9bd20c4e52a472a042f156ae3c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:14.847125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:14.857731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:14.883260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 832/1636] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:22:14.975930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41c1f9bd20c4e52a472a042f156ae3c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:14.996507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41c1f9bd20c4e52a472a042f156ae3c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:14.998759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41c1f9bd20c4e52a472a042f156ae3c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:15.023927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41c1f9bd20c4e52a472a042f156ae3c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:15.025078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d5b10854-26e8-430f-b6bc-46b4f717c178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41c1f9bd20c4e52a472a042f156ae3c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:15.060041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 805/1637] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:14 2022] DELETE /identity/v3/users/c41c1f9bd20c4e52a472a042f156ae3c => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:22:15.071496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46230335-ea34-4bba-a656-ed354ce7d837 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:15.072446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46230335-ea34-4bba-a656-ed354ce7d837 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:15.073601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46230335-ea34-4bba-a656-ed354ce7d837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:15.124734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-46230335-ea34-4bba-a656-ed354ce7d837 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:15.128945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-46230335-ea34-4bba-a656-ed354ce7d837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:15.347202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 833/1638] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:15.682873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b9db-7c8e-40da-b55a-01531e4ee9cc None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:15.683773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b9db-7c8e-40da-b55a-01531e4ee9cc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:15.683773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b9db-7c8e-40da-b55a-01531e4ee9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:15.719238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:15.770864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1a43b9db-7c8e-40da-b55a-01531e4ee9cc None None] MFA Rules not processed for user `a5fee90ee6694ae4a821f60ca194ce84`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:15.774380 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1a43b9db-7c8e-40da-b55a-01531e4ee9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:16.037565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 806/1639] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:16.039090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e814a45-11fe-49b5-9a9d-b11683f1c567 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:16.039835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e814a45-11fe-49b5-9a9d-b11683f1c567 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:16.039835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e814a45-11fe-49b5-9a9d-b11683f1c567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:16.082308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4e814a45-11fe-49b5-9a9d-b11683f1c567 None None] MFA Rules not processed for user `a5fee90ee6694ae4a821f60ca194ce84`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:16.086852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4e814a45-11fe-49b5-9a9d-b11683f1c567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:16.110301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 834/1640] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:16.124396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b9db-7c8e-40da-b55a-01531e4ee9cc None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:16.126841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1a43b9db-7c8e-40da-b55a-01531e4ee9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:16.155091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e814a45-11fe-49b5-9a9d-b11683f1c567 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:16.157049 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4e814a45-11fe-49b5-9a9d-b11683f1c567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:16.355960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e814a45-11fe-49b5-9a9d-b11683f1c567 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:16.370151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87807acc-9e2a-4489-b410-ac955912ef44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:16.370849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87807acc-9e2a-4489-b410-ac955912ef44 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:16.371140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87807acc-9e2a-4489-b410-ac955912ef44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:16.371423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87807acc-9e2a-4489-b410-ac955912ef44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:16.390062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b9db-7c8e-40da-b55a-01531e4ee9cc None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:16.396162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87807acc-9e2a-4489-b410-ac955912ef44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:16.400193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f22a4-8a1a-4a0f-a295-7d51dc3be0cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:16.400583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87807acc-9e2a-4489-b410-ac955912ef44 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:16.401000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f22a4-8a1a-4a0f-a295-7d51dc3be0cc service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:16.401266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f22a4-8a1a-4a0f-a295-7d51dc3be0cc service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:16.401690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f22a4-8a1a-4a0f-a295-7d51dc3be0cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:16.430035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f22a4-8a1a-4a0f-a295-7d51dc3be0cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:16.431706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f22a4-8a1a-4a0f-a295-7d51dc3be0cc service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:16.445508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 835/1641] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:22:16.449873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 807/1642] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:18.466691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e1617-38b4-4aed-be1f-c5615da0f2f8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:18.467675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e1617-38b4-4aed-be1f-c5615da0f2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:18.467675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e1617-38b4-4aed-be1f-c5615da0f2f8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:18.470124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 836/1643] 10.222.0.13 () {54 vars in 912 bytes} [Mon Jun 6 14:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:22:18.491272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87836557-dbb0-4896-ba3e-742b91445f37 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:18.491963 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87836557-dbb0-4896-ba3e-742b91445f37 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:18.492973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87836557-dbb0-4896-ba3e-742b91445f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:18.692011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-87836557-dbb0-4896-ba3e-742b91445f37 None None] MFA Rules not processed for user `59cd9d922204496d8ea185716b832481`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:18.694171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87836557-dbb0-4896-ba3e-742b91445f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:18.729442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 808/1644] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:22:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:18.772201 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e1617-38b4-4aed-be1f-c5615da0f2f8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:18.775248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8f1e1617-38b4-4aed-be1f-c5615da0f2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:18.825229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e1617-38b4-4aed-be1f-c5615da0f2f8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:18.842179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d299fa-2e78-43c8-991c-816ee1fa5965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:18.843842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d299fa-2e78-43c8-991c-816ee1fa5965 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:18.844724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d299fa-2e78-43c8-991c-816ee1fa5965 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:18.845509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d299fa-2e78-43c8-991c-816ee1fa5965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:18.865037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d299fa-2e78-43c8-991c-816ee1fa5965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:18.868669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d299fa-2e78-43c8-991c-816ee1fa5965 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:18.918210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 837/1645] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:19.260020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87836557-dbb0-4896-ba3e-742b91445f37 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:19.264692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87836557-dbb0-4896-ba3e-742b91445f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:19.293386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d299fa-2e78-43c8-991c-816ee1fa5965 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:19.295018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f2d299fa-2e78-43c8-991c-816ee1fa5965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:19.331034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d299fa-2e78-43c8-991c-816ee1fa5965 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:19.353409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87836557-dbb0-4896-ba3e-742b91445f37 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:19.359716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:19.361623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:19.362536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:19.363074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] PATH_INFO: `/v3/projects/001d77c3bc65465dbaa77d48cdb08b0a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:19.368709 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=001d77c3bc65465dbaa77d48cdb08b0a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:19.382070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:19.383113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:19.384274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:19.384704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:19.385114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] PATH_INFO: `/v3/users/7c73fcf874e2461e920859b5c3ee6599` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:19.387848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c73fcf874e2461e920859b5c3ee6599)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:19.390230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:19.527378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001d77c3bc65465dbaa77d48cdb08b0a', 'request_id': 'req-4d89f23e-4422-4631-8ffc-131562b60512'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.553550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001d77c3bc65465dbaa77d48cdb08b0a', 'request_id': 'req-4d89f23e-4422-4631-8ffc-131562b60512'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.567703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c73fcf874e2461e920859b5c3ee6599'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.571450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001d77c3bc65465dbaa77d48cdb08b0a', 'request_id': 'req-4d89f23e-4422-4631-8ffc-131562b60512'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.593979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c73fcf874e2461e920859b5c3ee6599'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.595556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c73fcf874e2461e920859b5c3ee6599'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.598103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:19.603794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] The token cache is being invalidate because project 001d77c3bc65465dbaa77d48cdb08b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:19.607016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.611017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.626997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 838/1646] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:19 2022] DELETE /identity/v3/projects/001d77c3bc65465dbaa77d48cdb08b0a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:22:19.629412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:19.633305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:19.652673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:19.667411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c73fcf874e2461e920859b5c3ee6599'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.675512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-570b07d6-8af7-4d34-8147-e773aeb89cfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c73fcf874e2461e920859b5c3ee6599'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:19.724912 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 809/1647] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:19 2022] DELETE /identity/v3/users/7c73fcf874e2461e920859b5c3ee6599 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:19.733945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000ddce-ec0a-4faa-b81c-ebb959973a4b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:19.734382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000ddce-ec0a-4faa-b81c-ebb959973a4b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:19.737501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000ddce-ec0a-4faa-b81c-ebb959973a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:19.857086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1000ddce-ec0a-4faa-b81c-ebb959973a4b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:19.859675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1000ddce-ec0a-4faa-b81c-ebb959973a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:19.931143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89f23e-4422-4631-8ffc-131562b60512 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:20.151342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 810/1648] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:20.200944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000ddce-ec0a-4faa-b81c-ebb959973a4b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:20.203253 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1000ddce-ec0a-4faa-b81c-ebb959973a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:20.237098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:20.242159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:20.242612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:20.243119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] PATH_INFO: `/v3/users/3d1cef9ba36046ca82d716796098954c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:20.246616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d1cef9ba36046ca82d716796098954c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:20.248723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:22:20.248723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: warnings.warn(msg) Jun 06 14:22:20.248723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: Jun 06 14:22:20.254424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:20.493683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000ddce-ec0a-4faa-b81c-ebb959973a4b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:20.511156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76003417-3ab5-4a95-879c-eaa838316f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:20.512238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003417-3ab5-4a95-879c-eaa838316f84 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:20.512700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003417-3ab5-4a95-879c-eaa838316f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:20.513143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003417-3ab5-4a95-879c-eaa838316f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:20.562616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d1cef9ba36046ca82d716796098954c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:20.573960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76003417-3ab5-4a95-879c-eaa838316f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:20.576701 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76003417-3ab5-4a95-879c-eaa838316f84 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:20.601893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d1cef9ba36046ca82d716796098954c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:20.603755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d1cef9ba36046ca82d716796098954c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:20.649034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d1cef9ba36046ca82d716796098954c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:20.649221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 811/1649] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:22:20.776889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9839c022-2595-4a3c-b671-ba19abfc3c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d1cef9ba36046ca82d716796098954c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:20.814688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 839/1650] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:19 2022] DELETE /identity/v3/users/3d1cef9ba36046ca82d716796098954c => generated 0 bytes in 1187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:20.941694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48a49c-e0fe-4c91-94d1-0c99517a0ae5 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:20.942688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48a49c-e0fe-4c91-94d1-0c99517a0ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:20.943163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48a49c-e0fe-4c91-94d1-0c99517a0ae5 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:20.947020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 812/1651] 10.222.0.13 () {54 vars in 960 bytes} [Mon Jun 6 14:22:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:22:20.954399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cf17d-5a5f-45b5-be5f-5fd5da1d1467 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:20.954731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cf17d-5a5f-45b5-be5f-5fd5da1d1467 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:20.955003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cf17d-5a5f-45b5-be5f-5fd5da1d1467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:20.994993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d23cf17d-5a5f-45b5-be5f-5fd5da1d1467 None None] MFA Rules not processed for user `51b91456a0634e0fa22fb389f0721a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:20.999689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d23cf17d-5a5f-45b5-be5f-5fd5da1d1467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:21.070484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:21.165762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 840/1652] 10.222.0.13 () {58 vars in 1057 bytes} [Mon Jun 6 14:22:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:21.175064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48a49c-e0fe-4c91-94d1-0c99517a0ae5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:21.216158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48a49c-e0fe-4c91-94d1-0c99517a0ae5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:21.243058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb7f11-a98e-4f6d-9155-4f265c61c4d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:21.244282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb7f11-a98e-4f6d-9155-4f265c61c4d1 service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:21.245378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb7f11-a98e-4f6d-9155-4f265c61c4d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:21.246137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb7f11-a98e-4f6d-9155-4f265c61c4d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:21.260207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2edb7f11-a98e-4f6d-9155-4f265c61c4d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:21.301803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb7f11-a98e-4f6d-9155-4f265c61c4d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:21.305184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb7f11-a98e-4f6d-9155-4f265c61c4d1 service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:21.365941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 813/1653] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:23.056375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cf17d-5a5f-45b5-be5f-5fd5da1d1467 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:23.059382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d23cf17d-5a5f-45b5-be5f-5fd5da1d1467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:23.313956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cf17d-5a5f-45b5-be5f-5fd5da1d1467 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:23.325503 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:23.327566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:23.328219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:23.329058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] PATH_INFO: `/v3/projects/7f970147e5094c01bacf052bbc47c129` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:23.332060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f970147e5094c01bacf052bbc47c129)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:23.337433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:23.462896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f970147e5094c01bacf052bbc47c129', 'request_id': 'req-12396698-1a4e-480c-84ba-43a3710674d1'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:23.475804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f970147e5094c01bacf052bbc47c129', 'request_id': 'req-12396698-1a4e-480c-84ba-43a3710674d1'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:23.486049 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f970147e5094c01bacf052bbc47c129', 'request_id': 'req-12396698-1a4e-480c-84ba-43a3710674d1'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:23.508813 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] The token cache is being invalidate because project 7f970147e5094c01bacf052bbc47c129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:23.509168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:23.510455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12396698-1a4e-480c-84ba-43a3710674d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:23.512670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 841/1654] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:23 2022] DELETE /identity/v3/projects/7f970147e5094c01bacf052bbc47c129 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:23.520698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54025f7-0cc9-4fa3-bf79-d3377c0cae91 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:23.521562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54025f7-0cc9-4fa3-bf79-d3377c0cae91 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:23.522073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54025f7-0cc9-4fa3-bf79-d3377c0cae91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:23.597418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e54025f7-0cc9-4fa3-bf79-d3377c0cae91 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:23.604689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e54025f7-0cc9-4fa3-bf79-d3377c0cae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:23.931495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 814/1655] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:23.945133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526b46a-a662-49fc-b9a6-30207b622126 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:23.945594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526b46a-a662-49fc-b9a6-30207b622126 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:23.945779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526b46a-a662-49fc-b9a6-30207b622126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:23.998930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5526b46a-a662-49fc-b9a6-30207b622126 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:24.004134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5526b46a-a662-49fc-b9a6-30207b622126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:24.038336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 842/1656] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:24.040362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54025f7-0cc9-4fa3-bf79-d3377c0cae91 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:24.071292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:24.102912 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54025f7-0cc9-4fa3-bf79-d3377c0cae91 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:24.144118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a9799-2344-4df6-9259-442000d2652a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:24.148480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a9799-2344-4df6-9259-442000d2652a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:24.149120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a9799-2344-4df6-9259-442000d2652a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:24.149441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a9799-2344-4df6-9259-442000d2652a admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:24.150911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a9799-2344-4df6-9259-442000d2652a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:24.164448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a9799-2344-4df6-9259-442000d2652a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:24.170482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a9799-2344-4df6-9259-442000d2652a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:24.253577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 815/1657] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:24.258978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526b46a-a662-49fc-b9a6-30207b622126 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:24.309395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526b46a-a662-49fc-b9a6-30207b622126 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:24.347346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfceed-93a5-44bc-b5d6-f34857606d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:24.347346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfceed-93a5-44bc-b5d6-f34857606d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:24.347346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfceed-93a5-44bc-b5d6-f34857606d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:24.347836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfceed-93a5-44bc-b5d6-f34857606d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:24.353044 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfceed-93a5-44bc-b5d6-f34857606d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:24.376290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfceed-93a5-44bc-b5d6-f34857606d16 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:24.559301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 843/1658] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:24 2022] POST /identity/v3/projects => generated 379 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:22:24.563122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a9799-2344-4df6-9259-442000d2652a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:24.606189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a9799-2344-4df6-9259-442000d2652a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:24.860065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5dc6d-ce32-440c-ad9c-7fee8eaf1faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:24.861562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5dc6d-ce32-440c-ad9c-7fee8eaf1faf admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:24.862956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5dc6d-ce32-440c-ad9c-7fee8eaf1faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:24.863830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5dc6d-ce32-440c-ad9c-7fee8eaf1faf admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:24.865084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5dc6d-ce32-440c-ad9c-7fee8eaf1faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:24.867505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5dc6d-ce32-440c-ad9c-7fee8eaf1faf admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:24.919216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 816/1659] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:24 2022] POST /identity/v3/users => generated 290 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:24.921240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfceed-93a5-44bc-b5d6-f34857606d16 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:24.970153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfceed-93a5-44bc-b5d6-f34857606d16 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:25.024266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f26c0-5b50-4276-87fa-811f170180e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:25.025402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f26c0-5b50-4276-87fa-811f170180e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:25.026490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f26c0-5b50-4276-87fa-811f170180e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:25.026980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f26c0-5b50-4276-87fa-811f170180e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:25.028635 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f26c0-5b50-4276-87fa-811f170180e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:25.039163 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f26c0-5b50-4276-87fa-811f170180e5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:25.085411 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 844/1660] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:22:25.086975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5dc6d-ce32-440c-ad9c-7fee8eaf1faf admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:25.133567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5dc6d-ce32-440c-ad9c-7fee8eaf1faf admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:25.179614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544314bf-e95c-42be-8e55-96c5781a3f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:25.180767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544314bf-e95c-42be-8e55-96c5781a3f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:25.197460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544314bf-e95c-42be-8e55-96c5781a3f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:25.197460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544314bf-e95c-42be-8e55-96c5781a3f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:25.204858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544314bf-e95c-42be-8e55-96c5781a3f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:25.209117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544314bf-e95c-42be-8e55-96c5781a3f0a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:25.261852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 817/1661] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:25.262121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f26c0-5b50-4276-87fa-811f170180e5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:25.302887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f26c0-5b50-4276-87fa-811f170180e5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:25.315885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fd04c-3ee0-451f-afad-046671e4126d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:25.317125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fd04c-3ee0-451f-afad-046671e4126d admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:25.320099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fd04c-3ee0-451f-afad-046671e4126d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:25.320565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fd04c-3ee0-451f-afad-046671e4126d admin admin] PATH_INFO: `/v3/projects/4c8ae3e6ee684fd981fca434e5c3fb61/users/6ff14bdc344c496fb849bb8b37198eba/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:25.353393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fd04c-3ee0-451f-afad-046671e4126d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8ae3e6ee684fd981fca434e5c3fb61, user_id=6ff14bdc344c496fb849bb8b37198eba, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:25.363153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fd04c-3ee0-451f-afad-046671e4126d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:25.371210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:25.393515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 845/1662] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:25 2022] PUT /identity/v3/projects/4c8ae3e6ee684fd981fca434e5c3fb61/users/6ff14bdc344c496fb849bb8b37198eba/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:25.418403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1d3e0-2345-4416-87ab-60d0bf1cbc5b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:25.418918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1d3e0-2345-4416-87ab-60d0bf1cbc5b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:25.419406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1d3e0-2345-4416-87ab-60d0bf1cbc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:25.517104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-57e1d3e0-2345-4416-87ab-60d0bf1cbc5b None None] MFA Rules not processed for user `6ff14bdc344c496fb849bb8b37198eba`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:25.518778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-57e1d3e0-2345-4416-87ab-60d0bf1cbc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:25.520742 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fd04c-3ee0-451f-afad-046671e4126d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:25.523634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-742fd04c-3ee0-451f-afad-046671e4126d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:25.754787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 818/1663] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:25.756163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1d3e0-2345-4416-87ab-60d0bf1cbc5b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:25.779488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fd04c-3ee0-451f-afad-046671e4126d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:25.792013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1d3e0-2345-4416-87ab-60d0bf1cbc5b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:25.800083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:25.801271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:25.801983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:25.802219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] PATH_INFO: `/v3/projects/3ea67b48983d49b8a5a12c85e5d28715` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:25.804378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7dd72-3e09-46d3-ae03-1b285ba61f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:25.805000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7dd72-3e09-46d3-ae03-1b285ba61f3c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:25.805149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ea67b48983d49b8a5a12c85e5d28715)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:25.805284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7dd72-3e09-46d3-ae03-1b285ba61f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:25.805652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7dd72-3e09-46d3-ae03-1b285ba61f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:25.806292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7dd72-3e09-46d3-ae03-1b285ba61f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:25.808139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:25.812854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7dd72-3e09-46d3-ae03-1b285ba61f3c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:25.902378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 819/1664] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:25 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:25.903499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7dd72-3e09-46d3-ae03-1b285ba61f3c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:25.951977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea67b48983d49b8a5a12c85e5d28715', 'request_id': 'req-1e8b9527-00a6-4a94-92ee-df81ac350b26'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:25.988726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea67b48983d49b8a5a12c85e5d28715', 'request_id': 'req-1e8b9527-00a6-4a94-92ee-df81ac350b26'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:26.015498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea67b48983d49b8a5a12c85e5d28715', 'request_id': 'req-1e8b9527-00a6-4a94-92ee-df81ac350b26'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:26.062166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] The token cache is being invalidate because project 3ea67b48983d49b8a5a12c85e5d28715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:26.062608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:26.066414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1e8b9527-00a6-4a94-92ee-df81ac350b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:26.072916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 846/1665] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:25 2022] DELETE /identity/v3/projects/3ea67b48983d49b8a5a12c85e5d28715 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:26.076041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b8fe1-ff3d-45e8-bffa-5571d2d2565b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:26.076560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b8fe1-ff3d-45e8-bffa-5571d2d2565b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:26.077073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b8fe1-ff3d-45e8-bffa-5571d2d2565b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:26.158114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b83b8fe1-ff3d-45e8-bffa-5571d2d2565b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:26.168184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b83b8fe1-ff3d-45e8-bffa-5571d2d2565b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:26.417808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:26.544240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 847/1666] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:26.557446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab2c2b-b90c-43b9-8a98-cce577d166b0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:26.558144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab2c2b-b90c-43b9-8a98-cce577d166b0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:26.558641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab2c2b-b90c-43b9-8a98-cce577d166b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:26.592639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7dd72-3e09-46d3-ae03-1b285ba61f3c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:26.647083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9cab2c2b-b90c-43b9-8a98-cce577d166b0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:26.648679 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9cab2c2b-b90c-43b9-8a98-cce577d166b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:26.651078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011ff64-125b-4c1c-9074-0844501782a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:26.657015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ff64-125b-4c1c-9074-0844501782a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:26.661534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ff64-125b-4c1c-9074-0844501782a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:26.662386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ff64-125b-4c1c-9074-0844501782a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:26.670740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 848/1667] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:26.675237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab2c2b-b90c-43b9-8a98-cce577d166b0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:26.678140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011ff64-125b-4c1c-9074-0844501782a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:26.691299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011ff64-125b-4c1c-9074-0844501782a7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:26.696805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab2c2b-b90c-43b9-8a98-cce577d166b0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:26.714689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280ec27-1a0f-470e-883c-7d0bc61652ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:26.715829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280ec27-1a0f-470e-883c-7d0bc61652ee admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:26.716349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280ec27-1a0f-470e-883c-7d0bc61652ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:26.716991 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280ec27-1a0f-470e-883c-7d0bc61652ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:26.718156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280ec27-1a0f-470e-883c-7d0bc61652ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:26.718688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280ec27-1a0f-470e-883c-7d0bc61652ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:26.720646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280ec27-1a0f-470e-883c-7d0bc61652ee admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:26.757027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 849/1668] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:26.758365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280ec27-1a0f-470e-883c-7d0bc61652ee admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:26.771824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:26.784168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 820/1669] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:25 2022] POST /identity/v3/users => generated 297 bytes in 881 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:26.784905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280ec27-1a0f-470e-883c-7d0bc61652ee admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:26.788418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011ff64-125b-4c1c-9074-0844501782a7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:26.798864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0011ff64-125b-4c1c-9074-0844501782a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:26.810552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecd546-660f-4616-b763-d1a12d1840e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:26.811945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecd546-660f-4616-b763-d1a12d1840e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:26.812243 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecd546-660f-4616-b763-d1a12d1840e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:26.812540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecd546-660f-4616-b763-d1a12d1840e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:26.813288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecd546-660f-4616-b763-d1a12d1840e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:26.815657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecd546-660f-4616-b763-d1a12d1840e3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:26.832025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011ff64-125b-4c1c-9074-0844501782a7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:26.856509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb9909-16ec-4437-9d2e-ee29f8879e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:26.857653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb9909-16ec-4437-9d2e-ee29f8879e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:26.857973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb9909-16ec-4437-9d2e-ee29f8879e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:26.858251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb9909-16ec-4437-9d2e-ee29f8879e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:26.859080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb9909-16ec-4437-9d2e-ee29f8879e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:26.863758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb9909-16ec-4437-9d2e-ee29f8879e3a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:26.889033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 850/1670] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:26 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:26.893539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecd546-660f-4616-b763-d1a12d1840e3 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:26.902734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 821/1671] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:26.904124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb9909-16ec-4437-9d2e-ee29f8879e3a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:26.924433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb9909-16ec-4437-9d2e-ee29f8879e3a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:27.090829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecd546-660f-4616-b763-d1a12d1840e3 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:27.121312 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780c645-a360-4f09-8d8f-c4af3f099ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:27.122381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780c645-a360-4f09-8d8f-c4af3f099ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:27.122738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780c645-a360-4f09-8d8f-c4af3f099ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:27.123084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780c645-a360-4f09-8d8f-c4af3f099ba0 admin admin] PATH_INFO: `/v3/projects/417bb45454b94b3dbb2e8f146a325ff6/users/5c96f02dbe004aee90aa0620f8b437a6/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:27.123235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014c24e-6313-4d51-acc2-9970f55afee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:27.124238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014c24e-6313-4d51-acc2-9970f55afee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:27.124730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014c24e-6313-4d51-acc2-9970f55afee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:27.125195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014c24e-6313-4d51-acc2-9970f55afee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:27.126807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014c24e-6313-4d51-acc2-9970f55afee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:27.130456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014c24e-6313-4d51-acc2-9970f55afee3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:27.180949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780c645-a360-4f09-8d8f-c4af3f099ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417bb45454b94b3dbb2e8f146a325ff6, user_id=5c96f02dbe004aee90aa0620f8b437a6, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:27.205072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780c645-a360-4f09-8d8f-c4af3f099ba0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:27.210112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 851/1672] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:26 2022] POST /identity/v3/users => generated 287 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:27.212460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b5f40-fc13-4fda-870a-7566563287ae None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:27.213031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b5f40-fc13-4fda-870a-7566563287ae None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:27.213452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b5f40-fc13-4fda-870a-7566563287ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:27.252003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 822/1673] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:26 2022] PUT /identity/v3/projects/417bb45454b94b3dbb2e8f146a325ff6/users/5c96f02dbe004aee90aa0620f8b437a6/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:27.253439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780c645-a360-4f09-8d8f-c4af3f099ba0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:27.284563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-903b5f40-fc13-4fda-870a-7566563287ae None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:27.287177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-903b5f40-fc13-4fda-870a-7566563287ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:27.512679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780c645-a360-4f09-8d8f-c4af3f099ba0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:27.519876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 852/1674] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:22:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:27.520835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b5f40-fc13-4fda-870a-7566563287ae None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:27.540877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97c3a-ef12-4a5a-8dc8-45e742a57361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:27.542290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97c3a-ef12-4a5a-8dc8-45e742a57361 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:27.542494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97c3a-ef12-4a5a-8dc8-45e742a57361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:27.543290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97c3a-ef12-4a5a-8dc8-45e742a57361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:27.544331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c97c3a-ef12-4a5a-8dc8-45e742a57361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:27.548781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b5f40-fc13-4fda-870a-7566563287ae None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:27.549381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c97c3a-ef12-4a5a-8dc8-45e742a57361 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:27.570140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5e22d-bca0-4156-b3c6-adfd973ff2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:27.571429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5e22d-bca0-4156-b3c6-adfd973ff2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:27.571945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5e22d-bca0-4156-b3c6-adfd973ff2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:27.574804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5e22d-bca0-4156-b3c6-adfd973ff2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:27.575966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5e22d-bca0-4156-b3c6-adfd973ff2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:27.583585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5e22d-bca0-4156-b3c6-adfd973ff2f0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:27.613021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 853/1675] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:27.616986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58751c09-c79c-4664-9f5b-878c71f05811 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:27.622407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58751c09-c79c-4664-9f5b-878c71f05811 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:27.624667 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58751c09-c79c-4664-9f5b-878c71f05811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:27.641223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 823/1676] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:27.647351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97c3a-ef12-4a5a-8dc8-45e742a57361 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:27.731794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-58751c09-c79c-4664-9f5b-878c71f05811 None None] MFA Rules not processed for user `53505ac8ab6142a6bf0c3d6fe1cc721f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:27.737561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-58751c09-c79c-4664-9f5b-878c71f05811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:27.739757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97c3a-ef12-4a5a-8dc8-45e742a57361 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:27.765150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc112-f87d-45e9-bb3d-cb59f2b2c315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:27.766943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc112-f87d-45e9-bb3d-cb59f2b2c315 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:27.767173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc112-f87d-45e9-bb3d-cb59f2b2c315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:27.767594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bc112-f87d-45e9-bb3d-cb59f2b2c315 admin admin] PATH_INFO: `/v3/projects/417bb45454b94b3dbb2e8f146a325ff6/users/5c96f02dbe004aee90aa0620f8b437a6/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:27.775722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883bc112-f87d-45e9-bb3d-cb59f2b2c315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417bb45454b94b3dbb2e8f146a325ff6, user_id=5c96f02dbe004aee90aa0620f8b437a6, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:27.789267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883bc112-f87d-45e9-bb3d-cb59f2b2c315 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:27.840521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 824/1677] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:27 2022] PUT /identity/v3/projects/417bb45454b94b3dbb2e8f146a325ff6/users/5c96f02dbe004aee90aa0620f8b437a6/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:27.842453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc112-f87d-45e9-bb3d-cb59f2b2c315 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:27.954534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:27.958566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:27.985036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 854/1678] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:27.986277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58751c09-c79c-4664-9f5b-878c71f05811 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:27.993537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-58751c09-c79c-4664-9f5b-878c71f05811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:28.039833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58751c09-c79c-4664-9f5b-878c71f05811 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:28.068702 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bc112-f87d-45e9-bb3d-cb59f2b2c315 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:28.081010 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860148ef-625f-4878-bcae-78cb42aa7515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:28.081749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860148ef-625f-4878-bcae-78cb42aa7515 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:28.082034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860148ef-625f-4878-bcae-78cb42aa7515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:28.082310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860148ef-625f-4878-bcae-78cb42aa7515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:28.083073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860148ef-625f-4878-bcae-78cb42aa7515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:28.084810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860148ef-625f-4878-bcae-78cb42aa7515 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:28.115625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 825/1679] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:28.115625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188f743-791d-4f6c-ac09-8ce64b5ffba7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:28.115625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188f743-791d-4f6c-ac09-8ce64b5ffba7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:28.115625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188f743-791d-4f6c-ac09-8ce64b5ffba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:28.174994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0188f743-791d-4f6c-ac09-8ce64b5ffba7 None None] MFA Rules not processed for user `5c96f02dbe004aee90aa0620f8b437a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:28.176260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0188f743-791d-4f6c-ac09-8ce64b5ffba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:28.276770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:28.278415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:28.278930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:28.279608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] PATH_INFO: `/v3/users/4e354954d5264bd199b0111f9f41efa2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:28.284410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e354954d5264bd199b0111f9f41efa2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:28.307266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:28.448820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e354954d5264bd199b0111f9f41efa2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:28.452017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 826/1680] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:28.453847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188f743-791d-4f6c-ac09-8ce64b5ffba7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:28.463905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0188f743-791d-4f6c-ac09-8ce64b5ffba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:28.475457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e354954d5264bd199b0111f9f41efa2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:28.477808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e354954d5264bd199b0111f9f41efa2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:28.494941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e354954d5264bd199b0111f9f41efa2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:28.534146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e354954d5264bd199b0111f9f41efa2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:28.542047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 855/1681] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:27 2022] DELETE /identity/v3/users/4e354954d5264bd199b0111f9f41efa2 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:28.543040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:28.551338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:28.760976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3df7eb-ea75-4e0d-b3e1-f7581ed2bdb5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:28.779395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:28.780003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:28.780290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:28.780597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:28.787992 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:28.822039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:28.861287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188f743-791d-4f6c-ac09-8ce64b5ffba7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:29.010850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:29.014079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:29.015161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:29.061836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 856/1682] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:29.069367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:29.070101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a35379-3dac-4c96-834c-71580a090363 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:29.072682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:29.074592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a35379-3dac-4c96-834c-71580a090363 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:29.075305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a35379-3dac-4c96-834c-71580a090363 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:29.076120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a35379-3dac-4c96-834c-71580a090363 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:29.081255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87a35379-3dac-4c96-834c-71580a090363 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:29.303636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe78668-f319-42a4-8ad1-9573f51d71a1 service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:29.323310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0f693-96e5-46b4-be4c-acabffa54272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:29.328595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0f693-96e5-46b4-be4c-acabffa54272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:29.329099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0f693-96e5-46b4-be4c-acabffa54272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:29.329597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0f693-96e5-46b4-be4c-acabffa54272 admin admin] PATH_INFO: `/v3/projects/b374dcf50b5a410b861f168ea10410e7/users/f8b41327fc2f463c9a4b1e516a4ca52a/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:29.391272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0f693-96e5-46b4-be4c-acabffa54272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b374dcf50b5a410b861f168ea10410e7, user_id=f8b41327fc2f463c9a4b1e516a4ca52a, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:29.400268 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0f693-96e5-46b4-be4c-acabffa54272 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:29.402825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a35379-3dac-4c96-834c-71580a090363 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:29.413079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a35379-3dac-4c96-834c-71580a090363 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:29.451433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 857/1683] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:29 2022] PUT /identity/v3/projects/b374dcf50b5a410b861f168ea10410e7/users/f8b41327fc2f463c9a4b1e516a4ca52a/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:29.453037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0f693-96e5-46b4-be4c-acabffa54272 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:29.454593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 827/1684] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:22:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1002 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:22:29.467046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780f769-6be6-49a7-bdd1-4dbbdec32b64 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:29.467837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780f769-6be6-49a7-bdd1-4dbbdec32b64 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:29.470744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780f769-6be6-49a7-bdd1-4dbbdec32b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:29.492136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cbc0f693-96e5-46b4-be4c-acabffa54272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:29.554875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8780f769-6be6-49a7-bdd1-4dbbdec32b64 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:29.558207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8780f769-6be6-49a7-bdd1-4dbbdec32b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:29.865796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 828/1685] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:29.873458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb33425-574e-46b6-85d8-4cfb096374bd None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:29.876326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb33425-574e-46b6-85d8-4cfb096374bd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:29.886532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb33425-574e-46b6-85d8-4cfb096374bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:29.943594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0f693-96e5-46b4-be4c-acabffa54272 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:29.960668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:29.961875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:29.962372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:29.962938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:29.994543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:30.017743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fbb33425-574e-46b6-85d8-4cfb096374bd None None] MFA Rules not processed for user `f8b41327fc2f463c9a4b1e516a4ca52a`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:30.019434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fbb33425-574e-46b6-85d8-4cfb096374bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:30.173041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:30.234267 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:30.237123 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:30.277363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 858/1686] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 825 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:30.294894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:30.314334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:30.518518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 829/1687] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 651 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:30.526130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb33425-574e-46b6-85d8-4cfb096374bd None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:30.632759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb33425-574e-46b6-85d8-4cfb096374bd None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:30.681133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb451d9-2766-44ab-beeb-02c78cd0d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:30.685135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb451d9-2766-44ab-beeb-02c78cd0d7ba admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:30.685651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb451d9-2766-44ab-beeb-02c78cd0d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:30.689797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb451d9-2766-44ab-beeb-02c78cd0d7ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:30.693660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb451d9-2766-44ab-beeb-02c78cd0d7ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:30.722934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb451d9-2766-44ab-beeb-02c78cd0d7ba admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:30.788626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9f5f4-a0f8-4ed6-add0-18a7aa7a58a3 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:30.813423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc727d38-6e9e-4443-b6f0-5374fcdbe4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:30.824877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc727d38-6e9e-4443-b6f0-5374fcdbe4df service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:30.825666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc727d38-6e9e-4443-b6f0-5374fcdbe4df service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:30.830999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc727d38-6e9e-4443-b6f0-5374fcdbe4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:30.840468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:30.910893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc727d38-6e9e-4443-b6f0-5374fcdbe4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:30.914358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc727d38-6e9e-4443-b6f0-5374fcdbe4df service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:30.920110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 830/1688] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:30 2022] POST /identity/v3/projects => generated 373 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:30.921045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb451d9-2766-44ab-beeb-02c78cd0d7ba admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:30.984884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 859/1689] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:31.002359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08890481-89ae-4d45-9ed8-27e8e9300613 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:31.002994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08890481-89ae-4d45-9ed8-27e8e9300613 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:31.003752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08890481-89ae-4d45-9ed8-27e8e9300613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:31.052776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb451d9-2766-44ab-beeb-02c78cd0d7ba admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:31.060760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05262b5c-cd77-474d-9b5a-de89b0466b62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:31.062071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05262b5c-cd77-474d-9b5a-de89b0466b62 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:31.062805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05262b5c-cd77-474d-9b5a-de89b0466b62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:31.063312 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05262b5c-cd77-474d-9b5a-de89b0466b62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:31.066233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-05262b5c-cd77-474d-9b5a-de89b0466b62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:31.116338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-08890481-89ae-4d45-9ed8-27e8e9300613 None None] MFA Rules not processed for user `59cd9d922204496d8ea185716b832481`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:31.118040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-08890481-89ae-4d45-9ed8-27e8e9300613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:31.277928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 860/1690] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Jun 6 14:22:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:31.278940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08890481-89ae-4d45-9ed8-27e8e9300613 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:31.296023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05262b5c-cd77-474d-9b5a-de89b0466b62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:31.302583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05262b5c-cd77-474d-9b5a-de89b0466b62 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:31.362663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 831/1691] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:31.480355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08890481-89ae-4d45-9ed8-27e8e9300613 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:31.498990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f48419-5ee1-49f1-8cfb-33099b44f0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:31.499705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f48419-5ee1-49f1-8cfb-33099b44f0cd admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:31.499989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f48419-5ee1-49f1-8cfb-33099b44f0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:31.500367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f48419-5ee1-49f1-8cfb-33099b44f0cd admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:31.501153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f48419-5ee1-49f1-8cfb-33099b44f0cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:31.506565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f48419-5ee1-49f1-8cfb-33099b44f0cd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:31.562185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 861/1692] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:31 2022] POST /identity/v3/users => generated 293 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:31.567841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05262b5c-cd77-474d-9b5a-de89b0466b62 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:31.591540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05262b5c-cd77-474d-9b5a-de89b0466b62 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:31.610921 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f48419-5ee1-49f1-8cfb-33099b44f0cd admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:31.615239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-09f48419-5ee1-49f1-8cfb-33099b44f0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:31.623902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f2fe1-3da6-4b41-8a39-f35157e03777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:31.628000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f2fe1-3da6-4b41-8a39-f35157e03777 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:31.628364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f2fe1-3da6-4b41-8a39-f35157e03777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:31.630282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f2fe1-3da6-4b41-8a39-f35157e03777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:31.631123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f2fe1-3da6-4b41-8a39-f35157e03777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:31.636725 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f2fe1-3da6-4b41-8a39-f35157e03777 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:31.676350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f48419-5ee1-49f1-8cfb-33099b44f0cd admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:31.698066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ba8e-13b9-42a9-b609-f8bbff7c30d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:31.701435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ba8e-13b9-42a9-b609-f8bbff7c30d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:31.702619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ba8e-13b9-42a9-b609-f8bbff7c30d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:31.705932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ba8e-13b9-42a9-b609-f8bbff7c30d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:31.732711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 832/1693] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:31.738234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f2fe1-3da6-4b41-8a39-f35157e03777 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:31.754245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ba8e-13b9-42a9-b609-f8bbff7c30d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:31.761297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ba8e-13b9-42a9-b609-f8bbff7c30d4 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:31.785778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 862/1694] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:31.812961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f2fe1-3da6-4b41-8a39-f35157e03777 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:31.867167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b2732-0c18-4df9-a240-6f9c4a7c603a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:31.878190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b2732-0c18-4df9-a240-6f9c4a7c603a admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:31.880966 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b2732-0c18-4df9-a240-6f9c4a7c603a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:31.881680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b2732-0c18-4df9-a240-6f9c4a7c603a admin admin] PATH_INFO: `/v3/projects/3c4b496657db4206b23d4eab1aec2d3b/users/7086881241454561bb17fe337919501c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:31.982001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b2732-0c18-4df9-a240-6f9c4a7c603a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4b496657db4206b23d4eab1aec2d3b, user_id=7086881241454561bb17fe337919501c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:32.000255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b2732-0c18-4df9-a240-6f9c4a7c603a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:32.075424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 833/1695] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:31 2022] PUT /identity/v3/projects/3c4b496657db4206b23d4eab1aec2d3b/users/7086881241454561bb17fe337919501c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:32.081507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ba8e-13b9-42a9-b609-f8bbff7c30d4 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:32.109664 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ba8e-13b9-42a9-b609-f8bbff7c30d4 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:32.163401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:32.375488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ae541-d48c-4c5e-97a5-2320f0d09419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:32.388143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ae541-d48c-4c5e-97a5-2320f0d09419 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:32.388143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ae541-d48c-4c5e-97a5-2320f0d09419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:32.388143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ae541-d48c-4c5e-97a5-2320f0d09419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:32.389952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ae541-d48c-4c5e-97a5-2320f0d09419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:32.397242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ae541-d48c-4c5e-97a5-2320f0d09419 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:32.448578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b2732-0c18-4df9-a240-6f9c4a7c603a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:32.450127 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 863/1696] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:32.476536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b2732-0c18-4df9-a240-6f9c4a7c603a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:32.551002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0847393-4456-47a5-a1b8-d8a2e26ce009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:32.571341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0847393-4456-47a5-a1b8-d8a2e26ce009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:32.580999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0847393-4456-47a5-a1b8-d8a2e26ce009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:32.585882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0847393-4456-47a5-a1b8-d8a2e26ce009 admin admin] PATH_INFO: `/v3/projects/3c4b496657db4206b23d4eab1aec2d3b/users/7086881241454561bb17fe337919501c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:32.613421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0847393-4456-47a5-a1b8-d8a2e26ce009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4b496657db4206b23d4eab1aec2d3b, user_id=7086881241454561bb17fe337919501c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:32.648022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0847393-4456-47a5-a1b8-d8a2e26ce009 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:32.759356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 834/1697] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:32 2022] PUT /identity/v3/projects/3c4b496657db4206b23d4eab1aec2d3b/users/7086881241454561bb17fe337919501c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:32.763429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c1666-88e7-400c-ad1d-14f380b300f0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:32.764018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c1666-88e7-400c-ad1d-14f380b300f0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:32.764641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c1666-88e7-400c-ad1d-14f380b300f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:32.940253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a83c1666-88e7-400c-ad1d-14f380b300f0 None None] MFA Rules not processed for user `7086881241454561bb17fe337919501c`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:32.945930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a83c1666-88e7-400c-ad1d-14f380b300f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:33.349610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0847393-4456-47a5-a1b8-d8a2e26ce009 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:33.352936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b0847393-4456-47a5-a1b8-d8a2e26ce009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:33.418732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 864/1698] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 651 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:33.436160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c1666-88e7-400c-ad1d-14f380b300f0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:33.774484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0847393-4456-47a5-a1b8-d8a2e26ce009 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:33.806068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:33.811502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:33.812010 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:33.814076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] PATH_INFO: `/v3/projects/42d8a0a3a9204fae93eb16754608827b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:33.823990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c1666-88e7-400c-ad1d-14f380b300f0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:33.826744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d8a0a3a9204fae93eb16754608827b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:33.828890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 14:22:33.828890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jun 06 14:22:33.828890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: Jun 06 14:22:33.839466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:33.850575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:33.874502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943f613-70b0-4888-85af-d1371534ef93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:33.879761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943f613-70b0-4888-85af-d1371534ef93 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:33.888978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943f613-70b0-4888-85af-d1371534ef93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:33.890738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943f613-70b0-4888-85af-d1371534ef93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:33.964210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943f613-70b0-4888-85af-d1371534ef93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:33.977074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943f613-70b0-4888-85af-d1371534ef93 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:34.061761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 865/1699] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:34.208711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d8a0a3a9204fae93eb16754608827b', 'request_id': 'req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:34.250896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d8a0a3a9204fae93eb16754608827b', 'request_id': 'req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:34.296655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d8a0a3a9204fae93eb16754608827b', 'request_id': 'req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:34.467132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] The token cache is being invalidate because project 42d8a0a3a9204fae93eb16754608827b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:34.468034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:34.471062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-05a3ee8b-6f15-4ed0-ac4e-6a894e9211ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:34.482059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 835/1700] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:33 2022] DELETE /identity/v3/projects/42d8a0a3a9204fae93eb16754608827b => generated 0 bytes in 1131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:22:34.484580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537dd27-8535-4149-a7bd-fe1e1b65a898 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:34.486215 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537dd27-8535-4149-a7bd-fe1e1b65a898 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:34.487076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537dd27-8535-4149-a7bd-fe1e1b65a898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:34.592490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d537dd27-8535-4149-a7bd-fe1e1b65a898 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:34.594242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d537dd27-8535-4149-a7bd-fe1e1b65a898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:34.799793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:34.877651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c7a2f-429b-4e38-bfba-f5e0af3909a9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:34.878177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c7a2f-429b-4e38-bfba-f5e0af3909a9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:34.879265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c7a2f-429b-4e38-bfba-f5e0af3909a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:34.898927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 866/1701] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:34.950400 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1d3c7a2f-429b-4e38-bfba-f5e0af3909a9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:34.952862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1d3c7a2f-429b-4e38-bfba-f5e0af3909a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:34.982594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537dd27-8535-4149-a7bd-fe1e1b65a898 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:34.997574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 836/1702] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:35.004524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537dd27-8535-4149-a7bd-fe1e1b65a898 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:35.022571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d354481-157e-424d-b1f6-6d8dea742c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:35.026847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d354481-157e-424d-b1f6-6d8dea742c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:35.028597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d354481-157e-424d-b1f6-6d8dea742c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:35.031415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d354481-157e-424d-b1f6-6d8dea742c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:35.038568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d354481-157e-424d-b1f6-6d8dea742c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:35.043963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d354481-157e-424d-b1f6-6d8dea742c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:35.048424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d354481-157e-424d-b1f6-6d8dea742c9a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:35.073349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 867/1703] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:35.077237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c7a2f-429b-4e38-bfba-f5e0af3909a9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:35.103294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c7a2f-429b-4e38-bfba-f5e0af3909a9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:35.124699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ded085-a8e4-4afc-92d6-96e7c60375ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:35.126206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ded085-a8e4-4afc-92d6-96e7c60375ad admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:35.126803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ded085-a8e4-4afc-92d6-96e7c60375ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:35.127311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ded085-a8e4-4afc-92d6-96e7c60375ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:35.128443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ded085-a8e4-4afc-92d6-96e7c60375ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:35.131537 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ded085-a8e4-4afc-92d6-96e7c60375ad admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:35.194055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d354481-157e-424d-b1f6-6d8dea742c9a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:35.197825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 837/1704] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:35 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:35.216716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d354481-157e-424d-b1f6-6d8dea742c9a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:35.522856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ded085-a8e4-4afc-92d6-96e7c60375ad admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:35.526146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e5ded085-a8e4-4afc-92d6-96e7c60375ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:35.532731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e8c8d-c245-47a2-8e46-2be8b658438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:35.534011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e8c8d-c245-47a2-8e46-2be8b658438b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:35.534555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e8c8d-c245-47a2-8e46-2be8b658438b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:35.535059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e8c8d-c245-47a2-8e46-2be8b658438b admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:35.536428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e8c8d-c245-47a2-8e46-2be8b658438b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:35.559967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ded085-a8e4-4afc-92d6-96e7c60375ad admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:35.575517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e8c8d-c245-47a2-8e46-2be8b658438b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:35.575791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:35.576676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:35.579180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:35.579627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] PATH_INFO: `/v3/users/f8b41327fc2f463c9a4b1e516a4ca52a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:35.582578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b41327fc2f463c9a4b1e516a4ca52a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:35.591887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:35.711147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 868/1705] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:35 2022] POST /identity/v3/users => generated 288 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:22:35.719009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e8c8d-c245-47a2-8e46-2be8b658438b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:35.755571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e8c8d-c245-47a2-8e46-2be8b658438b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:35.774486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978827b-6c5c-4d70-97a0-b97f9ae8114e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:35.777759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978827b-6c5c-4d70-97a0-b97f9ae8114e admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:35.778331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978827b-6c5c-4d70-97a0-b97f9ae8114e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:35.779791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978827b-6c5c-4d70-97a0-b97f9ae8114e admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:35.780805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978827b-6c5c-4d70-97a0-b97f9ae8114e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:35.783840 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978827b-6c5c-4d70-97a0-b97f9ae8114e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:35.806862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b41327fc2f463c9a4b1e516a4ca52a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:35.834786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 869/1706] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:22:35.844544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b41327fc2f463c9a4b1e516a4ca52a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:35.845632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b41327fc2f463c9a4b1e516a4ca52a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:35.858082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978827b-6c5c-4d70-97a0-b97f9ae8114e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:35.882454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978827b-6c5c-4d70-97a0-b97f9ae8114e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:35.892978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b41327fc2f463c9a4b1e516a4ca52a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:35.893934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d8edb21d-4b9e-4fc4-bc69-998621b9b93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b41327fc2f463c9a4b1e516a4ca52a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:35.912286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae221b-39a4-4973-a36e-13cff7bfea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:35.914033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae221b-39a4-4973-a36e-13cff7bfea28 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:35.914536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae221b-39a4-4973-a36e-13cff7bfea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:35.914988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae221b-39a4-4973-a36e-13cff7bfea28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:35.917175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae221b-39a4-4973-a36e-13cff7bfea28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:35.919588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae221b-39a4-4973-a36e-13cff7bfea28 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:35.928334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 838/1707] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:35 2022] DELETE /identity/v3/users/f8b41327fc2f463c9a4b1e516a4ca52a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:35.934164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68062e5e-ea61-401a-b0ef-88111c17b99f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:35.934734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68062e5e-ea61-401a-b0ef-88111c17b99f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:35.938816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68062e5e-ea61-401a-b0ef-88111c17b99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:35.953427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 870/1708] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:35.955132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae221b-39a4-4973-a36e-13cff7bfea28 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:35.957122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c8ae221b-39a4-4973-a36e-13cff7bfea28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:35.989199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae221b-39a4-4973-a36e-13cff7bfea28 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:36.072117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-68062e5e-ea61-401a-b0ef-88111c17b99f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:36.084359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-68062e5e-ea61-401a-b0ef-88111c17b99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:36.297624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:36.379338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4b93f-1b5b-458e-9827-9fcb22753cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:36.384209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4b93f-1b5b-458e-9827-9fcb22753cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:36.387824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4b93f-1b5b-458e-9827-9fcb22753cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:36.388420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4b93f-1b5b-458e-9827-9fcb22753cc4 admin admin] PATH_INFO: `/v3/projects/0e0374761b2b46ae84b2f77ab406fab1/users/9316bbf654cd412684a7f6ca1a8b0002/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:36.504367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4b93f-1b5b-458e-9827-9fcb22753cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0374761b2b46ae84b2f77ab406fab1, user_id=9316bbf654cd412684a7f6ca1a8b0002, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:36.526865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4b93f-1b5b-458e-9827-9fcb22753cc4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:36.577887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 839/1709] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 649 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:36.590081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 871/1710] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:35 2022] PUT /identity/v3/projects/0e0374761b2b46ae84b2f77ab406fab1/users/9316bbf654cd412684a7f6ca1a8b0002/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:36.596127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81e4c1-a265-4369-ad8b-b0d9bd0f6840 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:36.596650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81e4c1-a265-4369-ad8b-b0d9bd0f6840 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:36.597125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81e4c1-a265-4369-ad8b-b0d9bd0f6840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:36.841343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5f81e4c1-a265-4369-ad8b-b0d9bd0f6840 None None] MFA Rules not processed for user `9316bbf654cd412684a7f6ca1a8b0002`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:36.843884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5f81e4c1-a265-4369-ad8b-b0d9bd0f6840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:37.213126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 840/1711] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 619 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:37.219809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4b93f-1b5b-458e-9827-9fcb22753cc4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:37.274249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:37.519056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4b93f-1b5b-458e-9827-9fcb22753cc4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:37.540202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4859f0b-3ff1-4bd9-9cdd-2d6eca2fad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:37.542815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4859f0b-3ff1-4bd9-9cdd-2d6eca2fad7d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:37.543627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4859f0b-3ff1-4bd9-9cdd-2d6eca2fad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:37.544323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4859f0b-3ff1-4bd9-9cdd-2d6eca2fad7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:37.546658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4859f0b-3ff1-4bd9-9cdd-2d6eca2fad7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:37.562821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4859f0b-3ff1-4bd9-9cdd-2d6eca2fad7d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:37.706528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 872/1712] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:37 2022] POST /identity/v3/projects => generated 377 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:37.714242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81e4c1-a265-4369-ad8b-b0d9bd0f6840 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:37.754093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81e4c1-a265-4369-ad8b-b0d9bd0f6840 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:37.950437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4859f0b-3ff1-4bd9-9cdd-2d6eca2fad7d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:37.953601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c4859f0b-3ff1-4bd9-9cdd-2d6eca2fad7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:38.098586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664938f-668b-48f6-a741-c8aae00b6401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:38.099570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664938f-668b-48f6-a741-c8aae00b6401 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:38.100122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664938f-668b-48f6-a741-c8aae00b6401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:38.100559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664938f-668b-48f6-a741-c8aae00b6401 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:38.101482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664938f-668b-48f6-a741-c8aae00b6401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:38.106422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664938f-668b-48f6-a741-c8aae00b6401 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:38.215251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 841/1713] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:37 2022] POST /identity/v3/users => generated 295 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:38.220934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664938f-668b-48f6-a741-c8aae00b6401 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:38.254510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664938f-668b-48f6-a741-c8aae00b6401 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:38.296421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961840f8-7236-4b9c-866e-288c45f228ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:38.297311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961840f8-7236-4b9c-866e-288c45f228ba admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:38.297684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961840f8-7236-4b9c-866e-288c45f228ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:38.297944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961840f8-7236-4b9c-866e-288c45f228ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:38.298708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961840f8-7236-4b9c-866e-288c45f228ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:38.300391 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961840f8-7236-4b9c-866e-288c45f228ba admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:38.339928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 842/1714] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:22:38.341790 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a4cc9-f590-497c-b4d0-7bbb864b95ae None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:38.350357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a4cc9-f590-497c-b4d0-7bbb864b95ae None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:38.350965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a4cc9-f590-497c-b4d0-7bbb864b95ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:38.417311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4859f0b-3ff1-4bd9-9cdd-2d6eca2fad7d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:38.452123 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03aea82-5023-4f6f-8ec2-2dbfdf963695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:38.454169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03aea82-5023-4f6f-8ec2-2dbfdf963695 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:38.462639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03aea82-5023-4f6f-8ec2-2dbfdf963695 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:38.463539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03aea82-5023-4f6f-8ec2-2dbfdf963695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:38.464857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3d7a4cc9-f590-497c-b4d0-7bbb864b95ae None None] MFA Rules not processed for user `405ac2f6c07545efaa818e10fc788c05`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:38.467274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3d7a4cc9-f590-497c-b4d0-7bbb864b95ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:38.531601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03aea82-5023-4f6f-8ec2-2dbfdf963695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:38.535445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03aea82-5023-4f6f-8ec2-2dbfdf963695 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:38.678413 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 873/1715] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:38.686663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03aea82-5023-4f6f-8ec2-2dbfdf963695 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:38.739969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03aea82-5023-4f6f-8ec2-2dbfdf963695 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:38.777552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7c0ee-c9d3-48e0-9e48-9846c3ba6f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:38.781662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7c0ee-c9d3-48e0-9e48-9846c3ba6f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:38.784139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7c0ee-c9d3-48e0-9e48-9846c3ba6f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:38.793436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7c0ee-c9d3-48e0-9e48-9846c3ba6f3a admin admin] PATH_INFO: `/v3/projects/38a16b14fb274f398adc6f322131b5e2/users/02d3f706936d4cf58affbe98291b9093/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:38.811284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:38.879455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7c0ee-c9d3-48e0-9e48-9846c3ba6f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a16b14fb274f398adc6f322131b5e2, user_id=02d3f706936d4cf58affbe98291b9093, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:38.882824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7c0ee-c9d3-48e0-9e48-9846c3ba6f3a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:38.930057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 874/1716] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:38 2022] PUT /identity/v3/projects/38a16b14fb274f398adc6f322131b5e2/users/02d3f706936d4cf58affbe98291b9093/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:38.935179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7c0ee-c9d3-48e0-9e48-9846c3ba6f3a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:38.966439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 843/1717] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 625 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:39.041483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a4cc9-f590-497c-b4d0-7bbb864b95ae None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:39.045552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3d7a4cc9-f590-497c-b4d0-7bbb864b95ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:39.242216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7c0ee-c9d3-48e0-9e48-9846c3ba6f3a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:39.262767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70970361-937c-4b11-8a25-673caa74a3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:39.266732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70970361-937c-4b11-8a25-673caa74a3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:39.268474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70970361-937c-4b11-8a25-673caa74a3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:39.279876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70970361-937c-4b11-8a25-673caa74a3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:39.288326 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70970361-937c-4b11-8a25-673caa74a3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:39.294555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70970361-937c-4b11-8a25-673caa74a3a7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:39.342609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 875/1718] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:22:39.345112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70970361-937c-4b11-8a25-673caa74a3a7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:39.365744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70970361-937c-4b11-8a25-673caa74a3a7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:39.425305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1224e7-bae3-4ee0-ad68-813f9087ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:39.426376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1224e7-bae3-4ee0-ad68-813f9087ad63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:39.426671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1224e7-bae3-4ee0-ad68-813f9087ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:39.426941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1224e7-bae3-4ee0-ad68-813f9087ad63 admin admin] PATH_INFO: `/v3/projects/38a16b14fb274f398adc6f322131b5e2/users/02d3f706936d4cf58affbe98291b9093/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:39.441026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1224e7-bae3-4ee0-ad68-813f9087ad63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a16b14fb274f398adc6f322131b5e2, user_id=02d3f706936d4cf58affbe98291b9093, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:39.457603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1224e7-bae3-4ee0-ad68-813f9087ad63 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:39.548271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 876/1719] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:39 2022] PUT /identity/v3/projects/38a16b14fb274f398adc6f322131b5e2/users/02d3f706936d4cf58affbe98291b9093/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:39.556774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc92e3-ab41-4cdc-9925-7a2cd3b352f4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:39.558070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc92e3-ab41-4cdc-9925-7a2cd3b352f4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:39.558070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc92e3-ab41-4cdc-9925-7a2cd3b352f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:39.581397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:39.586061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a4cc9-f590-497c-b4d0-7bbb864b95ae None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:39.703604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4afc92e3-ab41-4cdc-9925-7a2cd3b352f4 None None] MFA Rules not processed for user `3f2e1c5d022c4bde9e76b8a933739557`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:39.705750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4afc92e3-ab41-4cdc-9925-7a2cd3b352f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:40.116672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 877/1720] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 558 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:40.116672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a9fc6-7f66-4a20-a0d9-22def7adaaad None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:40.116672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a9fc6-7f66-4a20-a0d9-22def7adaaad None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:40.116672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a9fc6-7f66-4a20-a0d9-22def7adaaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:40.138367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89243ab4-2f8a-49d3-8a34-d667fa03e062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:40.154671 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89243ab4-2f8a-49d3-8a34-d667fa03e062 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:40.154960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89243ab4-2f8a-49d3-8a34-d667fa03e062 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:40.155228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89243ab4-2f8a-49d3-8a34-d667fa03e062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:40.216923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-305a9fc6-7f66-4a20-a0d9-22def7adaaad None None] MFA Rules not processed for user `02d3f706936d4cf58affbe98291b9093`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:40.223452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-305a9fc6-7f66-4a20-a0d9-22def7adaaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:40.452512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 878/1721] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:40.453455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a9fc6-7f66-4a20-a0d9-22def7adaaad None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:40.460062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-305a9fc6-7f66-4a20-a0d9-22def7adaaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:40.583240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89243ab4-2f8a-49d3-8a34-d667fa03e062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:40.604099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89243ab4-2f8a-49d3-8a34-d667fa03e062 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:40.653574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 844/1722] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 1611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:40.656982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89243ab4-2f8a-49d3-8a34-d667fa03e062 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:40.666846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-89243ab4-2f8a-49d3-8a34-d667fa03e062 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:40.754378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a9fc6-7f66-4a20-a0d9-22def7adaaad None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:40.775685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:40.779544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:40.781239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:40.781564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] PATH_INFO: `/v3/projects/b374dcf50b5a410b861f168ea10410e7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:40.790652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b374dcf50b5a410b861f168ea10410e7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:40.814078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:40.817775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89243ab4-2f8a-49d3-8a34-d667fa03e062 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:40.867614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb9fd9-aa08-4e8d-b068-d6530d0e409f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:40.875881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb9fd9-aa08-4e8d-b068-d6530d0e409f service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:40.876431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb9fd9-aa08-4e8d-b068-d6530d0e409f service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:40.876949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb9fd9-aa08-4e8d-b068-d6530d0e409f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:40.944543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:40.949395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:40.985585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb9fd9-aa08-4e8d-b068-d6530d0e409f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:41.010019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb9fd9-aa08-4e8d-b068-d6530d0e409f service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:41.136386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 845/1723] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:41.140650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb9fd9-aa08-4e8d-b068-d6530d0e409f service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:41.158352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-eafb9fd9-aa08-4e8d-b068-d6530d0e409f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:41.175952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b374dcf50b5a410b861f168ea10410e7', 'request_id': 'req-382b4843-f0de-4854-af3d-fa3c28f525d3'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:41.221680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b374dcf50b5a410b861f168ea10410e7', 'request_id': 'req-382b4843-f0de-4854-af3d-fa3c28f525d3'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:41.280232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b374dcf50b5a410b861f168ea10410e7', 'request_id': 'req-382b4843-f0de-4854-af3d-fa3c28f525d3'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:41.341213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] The token cache is being invalidate because project b374dcf50b5a410b861f168ea10410e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:41.341533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:41.344183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:41.358924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 879/1724] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:40 2022] DELETE /identity/v3/projects/b374dcf50b5a410b861f168ea10410e7 => generated 0 bytes in 906 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:41.362853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:41.375023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:41.683697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb9fd9-aa08-4e8d-b068-d6530d0e409f service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:41.779978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b4843-f0de-4854-af3d-fa3c28f525d3 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:41.817306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:41.820700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:41.823426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:41.824997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] PATH_INFO: `/v3/users/7086881241454561bb17fe337919501c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:41.828821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7086881241454561bb17fe337919501c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:41.834077 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:41.956887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1593404-fd87-4929-ac31-24760e320cfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:41.957668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1593404-fd87-4929-ac31-24760e320cfa service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:41.957983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1593404-fd87-4929-ac31-24760e320cfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:41.958322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1593404-fd87-4929-ac31-24760e320cfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:41.972907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f1593404-fd87-4929-ac31-24760e320cfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:42.071769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7086881241454561bb17fe337919501c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:42.097022 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7086881241454561bb17fe337919501c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:42.098202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7086881241454561bb17fe337919501c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:42.121420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7086881241454561bb17fe337919501c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:42.178493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a80a573d-a081-4dfe-8673-dc261e4358a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7086881241454561bb17fe337919501c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:42.179473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1593404-fd87-4929-ac31-24760e320cfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:42.179473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1593404-fd87-4929-ac31-24760e320cfa service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:42.199562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 846/1725] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 1060 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:42.209007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 880/1726] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:41 2022] DELETE /identity/v3/users/7086881241454561bb17fe337919501c => generated 0 bytes in 847 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:42.985917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc584d-6d88-442f-b598-525cddfe56e3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:42.986605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc584d-6d88-442f-b598-525cddfe56e3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:42.987173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc584d-6d88-442f-b598-525cddfe56e3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:42.989294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 847/1727] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:22:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:22:43.349752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1382-f47f-4faf-b296-c75ffe637d5e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:43.349752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1382-f47f-4faf-b296-c75ffe637d5e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:43.349752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1382-f47f-4faf-b296-c75ffe637d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:43.493157 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-33ff1382-f47f-4faf-b296-c75ffe637d5e None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:43.498575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-33ff1382-f47f-4faf-b296-c75ffe637d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:43.616629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:43.863696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 881/1728] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:22:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 524 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:44.124454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc584d-6d88-442f-b598-525cddfe56e3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:44.126865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bdcc584d-6d88-442f-b598-525cddfe56e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:44.163027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff1382-f47f-4faf-b296-c75ffe637d5e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:44.166548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-33ff1382-f47f-4faf-b296-c75ffe637d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:44.241481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff1382-f47f-4faf-b296-c75ffe637d5e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:44.281640 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d681b09-61b2-4db9-a4bf-50c320553fb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:44.287529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d681b09-61b2-4db9-a4bf-50c320553fb0 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:44.289684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d681b09-61b2-4db9-a4bf-50c320553fb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:44.289993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d681b09-61b2-4db9-a4bf-50c320553fb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:44.326003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d681b09-61b2-4db9-a4bf-50c320553fb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:44.331026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d681b09-61b2-4db9-a4bf-50c320553fb0 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:44.379408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 882/1729] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:44.473272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc584d-6d88-442f-b598-525cddfe56e3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:44.500118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:44.506379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:44.508282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:44.511461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] PATH_INFO: `/v3/projects/3c4b496657db4206b23d4eab1aec2d3b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:44.527104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4b496657db4206b23d4eab1aec2d3b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:44.554247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:44.603374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:44.886935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4b496657db4206b23d4eab1aec2d3b', 'request_id': 'req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:44.914442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4b496657db4206b23d4eab1aec2d3b', 'request_id': 'req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:44.942032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4b496657db4206b23d4eab1aec2d3b', 'request_id': 'req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:45.047655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] The token cache is being invalidate because project 3c4b496657db4206b23d4eab1aec2d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:45.048275 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:45.053443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33bc6930-2d2d-4c4a-acd1-cf99e651bba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:45.068850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408ed87-d75b-4eaa-b789-3bc397aa166b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:45.069268 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408ed87-d75b-4eaa-b789-3bc397aa166b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:45.069771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408ed87-d75b-4eaa-b789-3bc397aa166b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:45.078045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 848/1730] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:44 2022] DELETE /identity/v3/projects/3c4b496657db4206b23d4eab1aec2d3b => generated 0 bytes in 936 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:22:45.128985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b408ed87-d75b-4eaa-b789-3bc397aa166b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:45.130747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b408ed87-d75b-4eaa-b789-3bc397aa166b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:45.265769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 883/1731] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:45.275291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5077-4892-425e-99fd-63f91675f628 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:45.276457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5077-4892-425e-99fd-63f91675f628 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:45.276992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5077-4892-425e-99fd-63f91675f628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:45.303891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-748a5077-4892-425e-99fd-63f91675f628 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:45.306936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-748a5077-4892-425e-99fd-63f91675f628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:45.319642 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 849/1732] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:45.325358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408ed87-d75b-4eaa-b789-3bc397aa166b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:45.338420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408ed87-d75b-4eaa-b789-3bc397aa166b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:45.344195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19defd98-088c-4e5d-8f3c-2387daf93081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:45.345350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19defd98-088c-4e5d-8f3c-2387daf93081 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:45.346383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19defd98-088c-4e5d-8f3c-2387daf93081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:45.346878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19defd98-088c-4e5d-8f3c-2387daf93081 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:45.348258 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19defd98-088c-4e5d-8f3c-2387daf93081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:45.348736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19defd98-088c-4e5d-8f3c-2387daf93081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:45.351655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19defd98-088c-4e5d-8f3c-2387daf93081 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:45.381050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5077-4892-425e-99fd-63f91675f628 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:45.399345 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 884/1733] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:45.405605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5077-4892-425e-99fd-63f91675f628 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:45.422794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceace8d-8c94-47d0-b483-68ce44fb80a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:45.423900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceace8d-8c94-47d0-b483-68ce44fb80a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:45.424674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceace8d-8c94-47d0-b483-68ce44fb80a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:45.425223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceace8d-8c94-47d0-b483-68ce44fb80a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:45.426531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceace8d-8c94-47d0-b483-68ce44fb80a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:45.434847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceace8d-8c94-47d0-b483-68ce44fb80a6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:45.505668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 850/1734] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:45 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:45.507082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19defd98-088c-4e5d-8f3c-2387daf93081 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:45.544319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19defd98-088c-4e5d-8f3c-2387daf93081 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:45.749940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca6ac2-adb3-4627-a2b8-04ba9ffd99b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:45.752771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca6ac2-adb3-4627-a2b8-04ba9ffd99b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:45.754897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca6ac2-adb3-4627-a2b8-04ba9ffd99b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:45.757660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca6ac2-adb3-4627-a2b8-04ba9ffd99b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:45.764084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca6ac2-adb3-4627-a2b8-04ba9ffd99b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:45.774785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca6ac2-adb3-4627-a2b8-04ba9ffd99b8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:45.963697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceace8d-8c94-47d0-b483-68ce44fb80a6 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:45.965231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 885/1735] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:45 2022] POST /identity/v3/users => generated 294 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:46.020585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceace8d-8c94-47d0-b483-68ce44fb80a6 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:46.050539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44b85e-7adf-4ad9-a322-b28b26724756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:46.061464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44b85e-7adf-4ad9-a322-b28b26724756 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:46.061464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44b85e-7adf-4ad9-a322-b28b26724756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:46.061464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44b85e-7adf-4ad9-a322-b28b26724756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:46.078036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44b85e-7adf-4ad9-a322-b28b26724756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:46.078036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44b85e-7adf-4ad9-a322-b28b26724756 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:46.152862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 851/1736] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:46.153183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca6ac2-adb3-4627-a2b8-04ba9ffd99b8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:46.205137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca6ac2-adb3-4627-a2b8-04ba9ffd99b8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:46.249616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779264f-07b1-4f69-bae6-3a0c6ade3403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:46.252237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779264f-07b1-4f69-bae6-3a0c6ade3403 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:46.252756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779264f-07b1-4f69-bae6-3a0c6ade3403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:46.253258 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779264f-07b1-4f69-bae6-3a0c6ade3403 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:46.259391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779264f-07b1-4f69-bae6-3a0c6ade3403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:46.273514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779264f-07b1-4f69-bae6-3a0c6ade3403 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:46.408451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 886/1737] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:22:46.409460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44b85e-7adf-4ad9-a322-b28b26724756 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:46.436996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44b85e-7adf-4ad9-a322-b28b26724756 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:46.462533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b0d31-ccc1-4a8d-ad38-8ee58e830cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b0d31-ccc1-4a8d-ad38-8ee58e830cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:46.465614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b0d31-ccc1-4a8d-ad38-8ee58e830cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:46.465958 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b0d31-ccc1-4a8d-ad38-8ee58e830cd8 admin admin] PATH_INFO: `/v3/projects/4d9784a13aed42f4a717299d910b7d2e/users/594054c9ef524e46a63616645eb4efe0/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:46.565439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b0d31-ccc1-4a8d-ad38-8ee58e830cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9784a13aed42f4a717299d910b7d2e, user_id=594054c9ef524e46a63616645eb4efe0, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:46.575776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b0d31-ccc1-4a8d-ad38-8ee58e830cd8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:46.650717 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 852/1738] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:46 2022] PUT /identity/v3/projects/4d9784a13aed42f4a717299d910b7d2e/users/594054c9ef524e46a63616645eb4efe0/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:22:46.677759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1109dd-e46a-4c31-abef-52e54e85f994 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:46.678355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1109dd-e46a-4c31-abef-52e54e85f994 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:46.679137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1109dd-e46a-4c31-abef-52e54e85f994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:46.756150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:46.772721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cf1109dd-e46a-4c31-abef-52e54e85f994 None None] MFA Rules not processed for user `594054c9ef524e46a63616645eb4efe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:46.774501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cf1109dd-e46a-4c31-abef-52e54e85f994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:47.018529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 887/1739] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:47.027354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b0d31-ccc1-4a8d-ad38-8ee58e830cd8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:47.271498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b0d31-ccc1-4a8d-ad38-8ee58e830cd8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:47.293220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85f0bc-9f74-4b16-98da-a845fb3757b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:47.294762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85f0bc-9f74-4b16-98da-a845fb3757b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:47.295744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85f0bc-9f74-4b16-98da-a845fb3757b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:47.296327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85f0bc-9f74-4b16-98da-a845fb3757b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:47.297655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85f0bc-9f74-4b16-98da-a845fb3757b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:47.302106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85f0bc-9f74-4b16-98da-a845fb3757b7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:47.406765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 853/1740] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:47 2022] POST /identity/v3/projects => generated 389 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:47.409098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1109dd-e46a-4c31-abef-52e54e85f994 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:47.492269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1109dd-e46a-4c31-abef-52e54e85f994 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:47.585979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85f0bc-9f74-4b16-98da-a845fb3757b7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:47.591493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4b85f0bc-9f74-4b16-98da-a845fb3757b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:47.656792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:47.756517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075095a9-0430-4bd6-913a-ee73ecd4ed42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:47.757289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075095a9-0430-4bd6-913a-ee73ecd4ed42 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:47.757572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075095a9-0430-4bd6-913a-ee73ecd4ed42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:47.762007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075095a9-0430-4bd6-913a-ee73ecd4ed42 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:47.763081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075095a9-0430-4bd6-913a-ee73ecd4ed42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:47.765738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075095a9-0430-4bd6-913a-ee73ecd4ed42 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:47.782250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85f0bc-9f74-4b16-98da-a845fb3757b7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:47.793653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:47.795512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:47.796202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:47.798367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] PATH_INFO: `/v3/users/6ff14bdc344c496fb849bb8b37198eba` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:47.808350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff14bdc344c496fb849bb8b37198eba)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:47.823042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:47.845381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 888/1741] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:47 2022] POST /identity/v3/users => generated 301 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:47.846460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075095a9-0430-4bd6-913a-ee73ecd4ed42 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:47.885295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075095a9-0430-4bd6-913a-ee73ecd4ed42 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:47.907383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bef56-ad87-4596-a06e-ae878f4bdd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:47.908105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bef56-ad87-4596-a06e-ae878f4bdd4d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:47.908387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bef56-ad87-4596-a06e-ae878f4bdd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:47.914319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bef56-ad87-4596-a06e-ae878f4bdd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:47.915140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bef56-ad87-4596-a06e-ae878f4bdd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:47.918866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bef56-ad87-4596-a06e-ae878f4bdd4d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:47.957488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 889/1742] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:47.959069 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bef56-ad87-4596-a06e-ae878f4bdd4d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:47.976421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bef56-ad87-4596-a06e-ae878f4bdd4d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:47.994954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faffb6f-f9a5-4685-9536-e264dc078014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:47.996289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faffb6f-f9a5-4685-9536-e264dc078014 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:47.996761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faffb6f-f9a5-4685-9536-e264dc078014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:47.997210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faffb6f-f9a5-4685-9536-e264dc078014 admin admin] PATH_INFO: `/v3/projects/ca31986f0dbd43089b1e6f51e648c34d/users/07cdc93aad7c4edfb0083f0b9431c47b/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:48.008446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff14bdc344c496fb849bb8b37198eba'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:48.025427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faffb6f-f9a5-4685-9536-e264dc078014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca31986f0dbd43089b1e6f51e648c34d, user_id=07cdc93aad7c4edfb0083f0b9431c47b, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:48.028804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faffb6f-f9a5-4685-9536-e264dc078014 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:48.048518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 890/1743] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:47 2022] PUT /identity/v3/projects/ca31986f0dbd43089b1e6f51e648c34d/users/07cdc93aad7c4edfb0083f0b9431c47b/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:22:48.052768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faffb6f-f9a5-4685-9536-e264dc078014 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:48.064032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff14bdc344c496fb849bb8b37198eba'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:48.084392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff14bdc344c496fb849bb8b37198eba'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:48.179269 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff14bdc344c496fb849bb8b37198eba'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:48.184225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4688d0cb-3cc6-4794-a16b-a904a38cf7ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff14bdc344c496fb849bb8b37198eba'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:48.230297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 854/1744] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:47 2022] DELETE /identity/v3/users/6ff14bdc344c496fb849bb8b37198eba => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:48.232238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a356f32-fab3-47f5-a0a0-19ddca71c84c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:48.232742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a356f32-fab3-47f5-a0a0-19ddca71c84c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:48.233154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a356f32-fab3-47f5-a0a0-19ddca71c84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:48.236471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:48.251002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faffb6f-f9a5-4685-9536-e264dc078014 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:48.298382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9a356f32-fab3-47f5-a0a0-19ddca71c84c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:48.299902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9a356f32-fab3-47f5-a0a0-19ddca71c84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:48.481142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b683b-2bbf-46b7-aa8b-5b93eeb09108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:48.481892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b683b-2bbf-46b7-aa8b-5b93eeb09108 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:48.482178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b683b-2bbf-46b7-aa8b-5b93eeb09108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:48.482542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b683b-2bbf-46b7-aa8b-5b93eeb09108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:48.486862 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b683b-2bbf-46b7-aa8b-5b93eeb09108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:48.488631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b683b-2bbf-46b7-aa8b-5b93eeb09108 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:48.491803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 855/1745] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:48.545697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 891/1746] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:48.550215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a356f32-fab3-47f5-a0a0-19ddca71c84c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:48.550215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9a356f32-fab3-47f5-a0a0-19ddca71c84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:48.557928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b683b-2bbf-46b7-aa8b-5b93eeb09108 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:48.559759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-018b683b-2bbf-46b7-aa8b-5b93eeb09108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:48.584087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a356f32-fab3-47f5-a0a0-19ddca71c84c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:48.605731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac05b6-769d-4edc-8a66-fd8581f47b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:48.606846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac05b6-769d-4edc-8a66-fd8581f47b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:48.607245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac05b6-769d-4edc-8a66-fd8581f47b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:48.607632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac05b6-769d-4edc-8a66-fd8581f47b44 admin admin] PATH_INFO: `/v3/projects/ca31986f0dbd43089b1e6f51e648c34d/users/07cdc93aad7c4edfb0083f0b9431c47b/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:48.611039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac05b6-769d-4edc-8a66-fd8581f47b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca31986f0dbd43089b1e6f51e648c34d, user_id=07cdc93aad7c4edfb0083f0b9431c47b, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:48.614068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac05b6-769d-4edc-8a66-fd8581f47b44 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:48.670967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 856/1747] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:48 2022] PUT /identity/v3/projects/ca31986f0dbd43089b1e6f51e648c34d/users/07cdc93aad7c4edfb0083f0b9431c47b/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:48.691868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f415f-2e0a-43ce-b498-89e3817a5fb3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:48.692344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f415f-2e0a-43ce-b498-89e3817a5fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:48.692767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f415f-2e0a-43ce-b498-89e3817a5fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:48.773214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-431f415f-2e0a-43ce-b498-89e3817a5fb3 None None] MFA Rules not processed for user `07cdc93aad7c4edfb0083f0b9431c47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:48.775208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-431f415f-2e0a-43ce-b498-89e3817a5fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:48.805382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:48.812246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b683b-2bbf-46b7-aa8b-5b93eeb09108 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:49.004619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43038371-1ae2-4743-a4a1-67de4aaedfe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:49.005603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43038371-1ae2-4743-a4a1-67de4aaedfe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:49.006103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43038371-1ae2-4743-a4a1-67de4aaedfe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:49.008684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43038371-1ae2-4743-a4a1-67de4aaedfe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:49.089199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 857/1748] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:49.098349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f415f-2e0a-43ce-b498-89e3817a5fb3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:49.105326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-431f415f-2e0a-43ce-b498-89e3817a5fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:49.238914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43038371-1ae2-4743-a4a1-67de4aaedfe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:49.247730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43038371-1ae2-4743-a4a1-67de4aaedfe6 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:49.294635 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 892/1749] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:49.296332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f415f-2e0a-43ce-b498-89e3817a5fb3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:49.317857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31946c83-ae3f-49d5-8a24-dfa9b45942af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:49.318507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31946c83-ae3f-49d5-8a24-dfa9b45942af service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:49.322052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31946c83-ae3f-49d5-8a24-dfa9b45942af service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:49.322870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31946c83-ae3f-49d5-8a24-dfa9b45942af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:49.391189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31946c83-ae3f-49d5-8a24-dfa9b45942af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:49.410243 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43038371-1ae2-4743-a4a1-67de4aaedfe6 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:49.411425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31946c83-ae3f-49d5-8a24-dfa9b45942af service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:49.411879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-43038371-1ae2-4743-a4a1-67de4aaedfe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:49.425660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:49.436199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43038371-1ae2-4743-a4a1-67de4aaedfe6 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:49.442726 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 858/1750] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:49.447768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a8d1b-4bb6-40df-823a-8206ef694059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:49.448753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a8d1b-4bb6-40df-823a-8206ef694059 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:49.449168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a8d1b-4bb6-40df-823a-8206ef694059 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:49.449456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a8d1b-4bb6-40df-823a-8206ef694059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:49.453960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-775a8d1b-4bb6-40df-823a-8206ef694059 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:49.792134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a8d1b-4bb6-40df-823a-8206ef694059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:49.811538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a8d1b-4bb6-40df-823a-8206ef694059 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:49.881131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31946c83-ae3f-49d5-8a24-dfa9b45942af service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:49.898852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 893/1751] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:22:49.907484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31946c83-ae3f-49d5-8a24-dfa9b45942af service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:49.925379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:49.927027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:49.928136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:49.928633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] PATH_INFO: `/v3/users/594054c9ef524e46a63616645eb4efe0` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:49.930657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=594054c9ef524e46a63616645eb4efe0)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:49.939681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:50.078024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '594054c9ef524e46a63616645eb4efe0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:50.097332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '594054c9ef524e46a63616645eb4efe0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:50.100736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '594054c9ef524e46a63616645eb4efe0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:50.135941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '594054c9ef524e46a63616645eb4efe0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:50.137489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0ed5757e-83cf-4565-af42-e73bc3f67c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '594054c9ef524e46a63616645eb4efe0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:50.186588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 859/1752] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:49 2022] DELETE /identity/v3/users/594054c9ef524e46a63616645eb4efe0 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:22:50.191739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a23b66-ba05-4c53-ae14-91fec97f8354 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:50.192036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a23b66-ba05-4c53-ae14-91fec97f8354 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:50.192454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a23b66-ba05-4c53-ae14-91fec97f8354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:50.256703 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e4a23b66-ba05-4c53-ae14-91fec97f8354 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:50.262575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e4a23b66-ba05-4c53-ae14-91fec97f8354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:50.573672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 894/1753] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:50.592919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a7f53-9158-4384-a48b-1c6b465b75f1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:50.593212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a7f53-9158-4384-a48b-1c6b465b75f1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:50.593496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a7f53-9158-4384-a48b-1c6b465b75f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:50.600677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a23b66-ba05-4c53-ae14-91fec97f8354 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:50.603577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e4a23b66-ba05-4c53-ae14-91fec97f8354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:50.692177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4a4a7f53-9158-4384-a48b-1c6b465b75f1 None None] MFA Rules not processed for user `86f9bcac527c45d6bc81b70e26544924`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:50.693739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4a4a7f53-9158-4384-a48b-1c6b465b75f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:50.753375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:50.838444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a23b66-ba05-4c53-ae14-91fec97f8354 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:50.849031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc6b68-c82a-45a9-a997-470c4e075b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:50.852573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc6b68-c82a-45a9-a997-470c4e075b30 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:50.855863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc6b68-c82a-45a9-a997-470c4e075b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:50.856515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc6b68-c82a-45a9-a997-470c4e075b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:50.890228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc6b68-c82a-45a9-a997-470c4e075b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:50.896165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc6b68-c82a-45a9-a997-470c4e075b30 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:50.935388 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 895/1754] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:50.966906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 860/1755] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:50.985220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8b45a-0430-4a59-b287-620bc59ae42c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:50.985515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8b45a-0430-4a59-b287-620bc59ae42c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:50.986119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8b45a-0430-4a59-b287-620bc59ae42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:51.030105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-81e8b45a-0430-4a59-b287-620bc59ae42c None None] MFA Rules not processed for user `86f9bcac527c45d6bc81b70e26544924`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:51.033473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-81e8b45a-0430-4a59-b287-620bc59ae42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:51.062195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 896/1756] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:51.079391 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a7f53-9158-4384-a48b-1c6b465b75f1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:51.081006 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4a4a7f53-9158-4384-a48b-1c6b465b75f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:51.240722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a7f53-9158-4384-a48b-1c6b465b75f1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:51.250328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5bc41-85b7-4ae1-bdfa-d206d7a63ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:51.251425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5bc41-85b7-4ae1-bdfa-d206d7a63ec2 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:51.251887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5bc41-85b7-4ae1-bdfa-d206d7a63ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:51.252314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5bc41-85b7-4ae1-bdfa-d206d7a63ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:51.277946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5bc41-85b7-4ae1-bdfa-d206d7a63ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:51.293767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5bc41-85b7-4ae1-bdfa-d206d7a63ec2 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:51.332192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 861/1757] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:51.966562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536acf4d-f708-4803-b200-3130f8786861 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:51.967492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536acf4d-f708-4803-b200-3130f8786861 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:51.968315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536acf4d-f708-4803-b200-3130f8786861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:52.011099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-536acf4d-f708-4803-b200-3130f8786861 None None] MFA Rules not processed for user `53505ac8ab6142a6bf0c3d6fe1cc721f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:52.012676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-536acf4d-f708-4803-b200-3130f8786861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:52.092513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:52.131648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337960-6c8b-4bca-8ffe-ec60ce1960ee None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:52.132644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337960-6c8b-4bca-8ffe-ec60ce1960ee None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:52.132644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337960-6c8b-4bca-8ffe-ec60ce1960ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:52.172250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 897/1758] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:52.185576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8d337960-6c8b-4bca-8ffe-ec60ce1960ee None None] MFA Rules not processed for user `405ac2f6c07545efaa818e10fc788c05`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:52.192920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8d337960-6c8b-4bca-8ffe-ec60ce1960ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:52.242986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3648f-ded3-4e4e-a74c-4d3e15995b29 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:52.243469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3648f-ded3-4e4e-a74c-4d3e15995b29 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:52.243944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3648f-ded3-4e4e-a74c-4d3e15995b29 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:52.246178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 898/1759] 10.222.0.13 () {54 vars in 912 bytes} [Mon Jun 6 14:22:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:22:52.260143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeca34e-3454-4701-98d4-7ce1799b6b9c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:52.260581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeca34e-3454-4701-98d4-7ce1799b6b9c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:52.261478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeca34e-3454-4701-98d4-7ce1799b6b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:52.303537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-abeca34e-3454-4701-98d4-7ce1799b6b9c None None] MFA Rules not processed for user `59cd9d922204496d8ea185716b832481`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:52.304886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-abeca34e-3454-4701-98d4-7ce1799b6b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:52.317766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 899/1760] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:22:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:52.344452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 862/1761] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:52.358587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeca34e-3454-4701-98d4-7ce1799b6b9c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:52.381450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeca34e-3454-4701-98d4-7ce1799b6b9c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:52.390509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883b6a9-d861-47d7-a0e6-b688b05c4046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:52.392017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883b6a9-d861-47d7-a0e6-b688b05c4046 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:52.392565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883b6a9-d861-47d7-a0e6-b688b05c4046 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:52.393011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883b6a9-d861-47d7-a0e6-b688b05c4046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:52.411506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883b6a9-d861-47d7-a0e6-b688b05c4046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:52.413314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883b6a9-d861-47d7-a0e6-b688b05c4046 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:52.419187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337960-6c8b-4bca-8ffe-ec60ce1960ee None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:52.420801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8d337960-6c8b-4bca-8ffe-ec60ce1960ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:52.425632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 900/1762] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:22:52.442360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337960-6c8b-4bca-8ffe-ec60ce1960ee None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:52.461156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:52.462207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:52.462623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:52.463112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] PATH_INFO: `/v3/projects/4c8ae3e6ee684fd981fca434e5c3fb61` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:52.464607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c8ae3e6ee684fd981fca434e5c3fb61)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:52.466820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:52.538476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:52.572652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c8ae3e6ee684fd981fca434e5c3fb61', 'request_id': 'req-f4c6013d-a206-4e41-866e-6ec770cce922'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:52.576808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c8ae3e6ee684fd981fca434e5c3fb61', 'request_id': 'req-f4c6013d-a206-4e41-866e-6ec770cce922'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:52.587994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c8ae3e6ee684fd981fca434e5c3fb61', 'request_id': 'req-f4c6013d-a206-4e41-866e-6ec770cce922'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:52.611576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] The token cache is being invalidate because project 4c8ae3e6ee684fd981fca434e5c3fb61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:52.612268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:52.614568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:52.617083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 863/1763] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:52 2022] DELETE /identity/v3/projects/4c8ae3e6ee684fd981fca434e5c3fb61 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:22:52.622908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883b6a9-d861-47d7-a0e6-b688b05c4046 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:52.661384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883b6a9-d861-47d7-a0e6-b688b05c4046 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:52.792849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:52.795095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:52.803789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:52.805052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:52.808210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:52.808906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] PATH_INFO: `/v3/users/5c96f02dbe004aee90aa0620f8b437a6` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:52.816411 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c96f02dbe004aee90aa0620f8b437a6)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:52.820678 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:52.841264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6013d-a206-4e41-866e-6ec770cce922 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:52.863523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:52.866312 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:52.867110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:52.868057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] PATH_INFO: `/v3/projects/4d9784a13aed42f4a717299d910b7d2e` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:52.883306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d9784a13aed42f4a717299d910b7d2e)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:52.887819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:52.950265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c96f02dbe004aee90aa0620f8b437a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:52.973917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c96f02dbe004aee90aa0620f8b437a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:52.976810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c96f02dbe004aee90aa0620f8b437a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.020196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c96f02dbe004aee90aa0620f8b437a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.064329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d9784a13aed42f4a717299d910b7d2e', 'request_id': 'req-755c6ef9-8f49-4c72-aec3-17ab62f104f1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.070682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-baa6dc8d-bb3c-437b-a07e-b490e2646544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c96f02dbe004aee90aa0620f8b437a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.087442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d9784a13aed42f4a717299d910b7d2e', 'request_id': 'req-755c6ef9-8f49-4c72-aec3-17ab62f104f1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.097085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 901/1764] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:52 2022] DELETE /identity/v3/users/5c96f02dbe004aee90aa0620f8b437a6 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:53.100397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7563e0-798b-4be4-a358-e0c2f8d63379 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:53.100842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7563e0-798b-4be4-a358-e0c2f8d63379 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:53.101159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7563e0-798b-4be4-a358-e0c2f8d63379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:53.101873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d9784a13aed42f4a717299d910b7d2e', 'request_id': 'req-755c6ef9-8f49-4c72-aec3-17ab62f104f1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.147642 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] The token cache is being invalidate because project 4d9784a13aed42f4a717299d910b7d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:53.148714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.153291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.158515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 864/1765] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:52 2022] DELETE /identity/v3/projects/4d9784a13aed42f4a717299d910b7d2e => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:53.164829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:53.175704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:53.176004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8a7563e0-798b-4be4-a358-e0c2f8d63379 None None] MFA Rules not processed for user `3f2e1c5d022c4bde9e76b8a933739557`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:53.178483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8a7563e0-798b-4be4-a358-e0c2f8d63379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:53.258673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:53.328474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 902/1766] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:53.341873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c6ef9-8f49-4c72-aec3-17ab62f104f1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:53.485747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:53.493819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:53.498021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:53.498491 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:53.498904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] PATH_INFO: `/v3/users/07cdc93aad7c4edfb0083f0b9431c47b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:53.501393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07cdc93aad7c4edfb0083f0b9431c47b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:53.505332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:53.606759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07cdc93aad7c4edfb0083f0b9431c47b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.625619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07cdc93aad7c4edfb0083f0b9431c47b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.626885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07cdc93aad7c4edfb0083f0b9431c47b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.654743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07cdc93aad7c4edfb0083f0b9431c47b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.656003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07cdc93aad7c4edfb0083f0b9431c47b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:53.686119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 865/1767] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:22:53 2022] DELETE /identity/v3/users/07cdc93aad7c4edfb0083f0b9431c47b => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:54.799041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7563e0-798b-4be4-a358-e0c2f8d63379 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:54.804560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8a7563e0-798b-4be4-a358-e0c2f8d63379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:54.860487 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:54.862194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7563e0-798b-4be4-a358-e0c2f8d63379 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:54.864083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:55.154628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a8a09-315b-4927-bb89-5addd0b33306 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:55.168906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:55.170297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:55.170660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:55.170955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] PATH_INFO: `/v3/projects/ca31986f0dbd43089b1e6f51e648c34d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:55.174242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca31986f0dbd43089b1e6f51e648c34d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:55.177047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:55.225573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:55.233660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:55.233660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:55.235347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] PATH_INFO: `/v3/projects/417bb45454b94b3dbb2e8f146a325ff6` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:55.238960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=417bb45454b94b3dbb2e8f146a325ff6)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:55.245865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:55.303320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca31986f0dbd43089b1e6f51e648c34d', 'request_id': 'req-f0d3494c-7705-48c6-b007-1cbe8584f182'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.333885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca31986f0dbd43089b1e6f51e648c34d', 'request_id': 'req-f0d3494c-7705-48c6-b007-1cbe8584f182'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.347078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca31986f0dbd43089b1e6f51e648c34d', 'request_id': 'req-f0d3494c-7705-48c6-b007-1cbe8584f182'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.390570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] The token cache is being invalidate because project ca31986f0dbd43089b1e6f51e648c34d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:55.390869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.393891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0d3494c-7705-48c6-b007-1cbe8584f182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.401649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 866/1768] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:54 2022] DELETE /identity/v3/projects/ca31986f0dbd43089b1e6f51e648c34d => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:55.403425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377f723-ddf9-4f15-8698-1de50dac29b4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:55.403714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377f723-ddf9-4f15-8698-1de50dac29b4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:55.403992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377f723-ddf9-4f15-8698-1de50dac29b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:55.419786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '417bb45454b94b3dbb2e8f146a325ff6', 'request_id': 'req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.440789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '417bb45454b94b3dbb2e8f146a325ff6', 'request_id': 'req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.445443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:55.474863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '417bb45454b94b3dbb2e8f146a325ff6', 'request_id': 'req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.475261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a377f723-ddf9-4f15-8698-1de50dac29b4 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:55.481275 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a377f723-ddf9-4f15-8698-1de50dac29b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:55.491171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] The token cache is being invalidate because project 417bb45454b94b3dbb2e8f146a325ff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:22:55.491519 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.492971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a9ba9b89-6c19-441e-8c7b-f6482c0a7a4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:22:55.510794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 903/1769] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:22:54 2022] DELETE /identity/v3/projects/417bb45454b94b3dbb2e8f146a325ff6 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:55.522761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10311c21-c936-42a3-8d7d-17cdcf88aaa1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:55.523100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10311c21-c936-42a3-8d7d-17cdcf88aaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:55.524746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10311c21-c936-42a3-8d7d-17cdcf88aaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:55.619478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-10311c21-c936-42a3-8d7d-17cdcf88aaa1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:55.621333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-10311c21-c936-42a3-8d7d-17cdcf88aaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:55.699752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 867/1770] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:55.704532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c05834-cdaf-41eb-bf26-ee22f72b7f83 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:55.704532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c05834-cdaf-41eb-bf26-ee22f72b7f83 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:55.704532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c05834-cdaf-41eb-bf26-ee22f72b7f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:55.761888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a6c05834-cdaf-41eb-bf26-ee22f72b7f83 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:55.766289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a6c05834-cdaf-41eb-bf26-ee22f72b7f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:55.794680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 868/1771] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:55.799467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c05834-cdaf-41eb-bf26-ee22f72b7f83 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:55.850302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c05834-cdaf-41eb-bf26-ee22f72b7f83 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:55.870992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb86371-19b1-4074-89be-23de8385e2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:55.872965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb86371-19b1-4074-89be-23de8385e2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:55.873583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb86371-19b1-4074-89be-23de8385e2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:55.874069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb86371-19b1-4074-89be-23de8385e2ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:55.877595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb86371-19b1-4074-89be-23de8385e2ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:55.878242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb86371-19b1-4074-89be-23de8385e2ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:55.880480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb86371-19b1-4074-89be-23de8385e2ca admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:55.905099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 869/1772] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:55.911012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb86371-19b1-4074-89be-23de8385e2ca admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:55.943616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb86371-19b1-4074-89be-23de8385e2ca admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:55.958116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 904/1773] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:55.964031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460f846-3b26-4922-ae16-29efdd86f176 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:55.964321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460f846-3b26-4922-ae16-29efdd86f176 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:55.964812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460f846-3b26-4922-ae16-29efdd86f176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:55.973610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ba474-632d-4af2-94ac-84ee51dfadc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:55.976464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ba474-632d-4af2-94ac-84ee51dfadc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.005819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c460f846-3b26-4922-ae16-29efdd86f176 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:56.007499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c460f846-3b26-4922-ae16-29efdd86f176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:56.007666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ba474-632d-4af2-94ac-84ee51dfadc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.007952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ba474-632d-4af2-94ac-84ee51dfadc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.010360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ba474-632d-4af2-94ac-84ee51dfadc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:56.022903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ba474-632d-4af2-94ac-84ee51dfadc2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:56.050222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 905/1774] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:22:56.054847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460f846-3b26-4922-ae16-29efdd86f176 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.072262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460f846-3b26-4922-ae16-29efdd86f176 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.096562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2e982-09dd-488f-8838-20ef1bad07b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.097259 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2e982-09dd-488f-8838-20ef1bad07b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.097542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2e982-09dd-488f-8838-20ef1bad07b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.097857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2e982-09dd-488f-8838-20ef1bad07b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.098617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2e982-09dd-488f-8838-20ef1bad07b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:22:56.099094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2e982-09dd-488f-8838-20ef1bad07b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:56.104170 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2e982-09dd-488f-8838-20ef1bad07b1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:56.108934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 870/1775] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:55 2022] POST /identity/v3/projects => generated 409 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:56.111583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ba474-632d-4af2-94ac-84ee51dfadc2 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.130520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:56.138166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 906/1776] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:22:56.144033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2e982-09dd-488f-8838-20ef1bad07b1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.159745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2e982-09dd-488f-8838-20ef1bad07b1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.246844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81360023-330b-4e01-bf7b-6c84d3dda333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.247524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81360023-330b-4e01-bf7b-6c84d3dda333 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.247957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81360023-330b-4e01-bf7b-6c84d3dda333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.248804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81360023-330b-4e01-bf7b-6c84d3dda333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.249591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81360023-330b-4e01-bf7b-6c84d3dda333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:56.252493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81360023-330b-4e01-bf7b-6c84d3dda333 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:56.267088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ba474-632d-4af2-94ac-84ee51dfadc2 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.274789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e7d6b-57c0-4d15-bf44-64fe6db0bf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.276211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e7d6b-57c0-4d15-bf44-64fe6db0bf31 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.276677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e7d6b-57c0-4d15-bf44-64fe6db0bf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.277139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e7d6b-57c0-4d15-bf44-64fe6db0bf31 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.278185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e7d6b-57c0-4d15-bf44-64fe6db0bf31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:56.280462 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e7d6b-57c0-4d15-bf44-64fe6db0bf31 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:56.304775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 907/1777] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:56 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:56.308658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81360023-330b-4e01-bf7b-6c84d3dda333 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.315228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 871/1778] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:56 2022] POST /identity/v3/users => generated 305 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:56.320468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e7d6b-57c0-4d15-bf44-64fe6db0bf31 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.385478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e7d6b-57c0-4d15-bf44-64fe6db0bf31 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.407358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81360023-330b-4e01-bf7b-6c84d3dda333 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.417106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a56e39-9070-455f-a8c0-0ac360a887eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.418124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a56e39-9070-455f-a8c0-0ac360a887eb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.418517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a56e39-9070-455f-a8c0-0ac360a887eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.418946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a56e39-9070-455f-a8c0-0ac360a887eb admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.419920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a56e39-9070-455f-a8c0-0ac360a887eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:56.422513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a56e39-9070-455f-a8c0-0ac360a887eb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:56.475649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 908/1779] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:56 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:22:56.476711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a56e39-9070-455f-a8c0-0ac360a887eb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.515642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a56e39-9070-455f-a8c0-0ac360a887eb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.542953 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:56.546582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dff113-8d06-4be1-a781-26fe04048319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.547438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dff113-8d06-4be1-a781-26fe04048319 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.547832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dff113-8d06-4be1-a781-26fe04048319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.548212 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dff113-8d06-4be1-a781-26fe04048319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.554398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dff113-8d06-4be1-a781-26fe04048319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:56.558237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dff113-8d06-4be1-a781-26fe04048319 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:56.569856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0af83-038a-4857-aac1-8ad8119dfa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.570626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0af83-038a-4857-aac1-8ad8119dfa83 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.570913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0af83-038a-4857-aac1-8ad8119dfa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.571288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0af83-038a-4857-aac1-8ad8119dfa83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.574628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0af83-038a-4857-aac1-8ad8119dfa83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:56.579309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0af83-038a-4857-aac1-8ad8119dfa83 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:56.605531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 909/1780] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:56.607211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b261-58db-41bc-a8c9-de4d3744a874 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.612059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 872/1781] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:56.613692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b261-58db-41bc-a8c9-de4d3744a874 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.615185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0af83-038a-4857-aac1-8ad8119dfa83 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.616098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b261-58db-41bc-a8c9-de4d3744a874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.648324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0af83-038a-4857-aac1-8ad8119dfa83 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.664628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5978c-6d7f-4e96-bffe-007d97f5a8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.666254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5978c-6d7f-4e96-bffe-007d97f5a8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.666545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5978c-6d7f-4e96-bffe-007d97f5a8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.666851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5978c-6d7f-4e96-bffe-007d97f5a8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.667756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5978c-6d7f-4e96-bffe-007d97f5a8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:56.669335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-aca8b261-58db-41bc-a8c9-de4d3744a874 None None] MFA Rules not processed for user `86f9bcac527c45d6bc81b70e26544924`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:56.669504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5978c-6d7f-4e96-bffe-007d97f5a8e1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:56.670890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-aca8b261-58db-41bc-a8c9-de4d3744a874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:56.706043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 873/1782] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:56.706655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5978c-6d7f-4e96-bffe-007d97f5a8e1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.726623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5978c-6d7f-4e96-bffe-007d97f5a8e1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.744654 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e84fb-5cd9-4678-8a23-8b10083f3cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.745397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e84fb-5cd9-4678-8a23-8b10083f3cca admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.746451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e84fb-5cd9-4678-8a23-8b10083f3cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.746451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e84fb-5cd9-4678-8a23-8b10083f3cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.748262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e84fb-5cd9-4678-8a23-8b10083f3cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:56.752667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e84fb-5cd9-4678-8a23-8b10083f3cca admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:56.830845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 874/1783] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:56.831397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e84fb-5cd9-4678-8a23-8b10083f3cca admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.895252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 910/1784] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:56.902804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b261-58db-41bc-a8c9-de4d3744a874 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:56.903097 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e84fb-5cd9-4678-8a23-8b10083f3cca admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.926935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:56.944336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a85c2-2c04-42e3-92ea-ac825a5329d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.947175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a85c2-2c04-42e3-92ea-ac825a5329d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.948801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a85c2-2c04-42e3-92ea-ac825a5329d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.950410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a85c2-2c04-42e3-92ea-ac825a5329d9 admin admin] PATH_INFO: `/v3/projects/a230399e40354b4896cf4e1dd1b3632c/users/0465889fa6144da09f9dca0fad2f15b8/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:56.950919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b261-58db-41bc-a8c9-de4d3744a874 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:56.973051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699160a2-7c19-4231-8e0f-6addf1ad7d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:56.974181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699160a2-7c19-4231-8e0f-6addf1ad7d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:56.974575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699160a2-7c19-4231-8e0f-6addf1ad7d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:56.974958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699160a2-7c19-4231-8e0f-6addf1ad7d7c admin admin] PATH_INFO: `/v3/projects/6b6a1ab356ae4dd7828d6010af911a4d/users/86a8b8b6e0da4cd4823cb7c8fcd48d84/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:57.038885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699160a2-7c19-4231-8e0f-6addf1ad7d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6a1ab356ae4dd7828d6010af911a4d, user_id=86a8b8b6e0da4cd4823cb7c8fcd48d84, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:57.039771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a85c2-2c04-42e3-92ea-ac825a5329d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a230399e40354b4896cf4e1dd1b3632c, user_id=0465889fa6144da09f9dca0fad2f15b8, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:57.043363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699160a2-7c19-4231-8e0f-6addf1ad7d7c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:57.052767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a85c2-2c04-42e3-92ea-ac825a5329d9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:57.074386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 911/1785] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:56 2022] PUT /identity/v3/projects/6b6a1ab356ae4dd7828d6010af911a4d/users/86a8b8b6e0da4cd4823cb7c8fcd48d84/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:57.085094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebf397-2282-4d94-91ee-bdc608da3362 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:57.091453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebf397-2282-4d94-91ee-bdc608da3362 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:57.092175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebf397-2282-4d94-91ee-bdc608da3362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:57.119644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 875/1786] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:56 2022] PUT /identity/v3/projects/a230399e40354b4896cf4e1dd1b3632c/users/0465889fa6144da09f9dca0fad2f15b8/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:57.131913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d32d64-9ef4-4531-92b2-a327b3343906 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:57.132353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d32d64-9ef4-4531-92b2-a327b3343906 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:57.132976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d32d64-9ef4-4531-92b2-a327b3343906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:57.160778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bfebf397-2282-4d94-91ee-bdc608da3362 None None] MFA Rules not processed for user `86a8b8b6e0da4cd4823cb7c8fcd48d84`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:57.162013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bfebf397-2282-4d94-91ee-bdc608da3362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:57.205663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-60d32d64-9ef4-4531-92b2-a327b3343906 None None] MFA Rules not processed for user `0465889fa6144da09f9dca0fad2f15b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:57.207924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-60d32d64-9ef4-4531-92b2-a327b3343906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:57.457859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 912/1787] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:57.459650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebf397-2282-4d94-91ee-bdc608da3362 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:57.663896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 876/1788] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 536 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:57.665490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d32d64-9ef4-4531-92b2-a327b3343906 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:58.000190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebf397-2282-4d94-91ee-bdc608da3362 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:58.037969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049c124-815d-4827-8949-2daaaefa5d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:58.040914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049c124-815d-4827-8949-2daaaefa5d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:58.049382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049c124-815d-4827-8949-2daaaefa5d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:58.050260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049c124-815d-4827-8949-2daaaefa5d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:58.060693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049c124-815d-4827-8949-2daaaefa5d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:58.069748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049c124-815d-4827-8949-2daaaefa5d23 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:58.164054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 913/1789] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:57 2022] POST /identity/v3/projects => generated 407 bytes in 706 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:58.170571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049c124-815d-4827-8949-2daaaefa5d23 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:58.215998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d32d64-9ef4-4531-92b2-a327b3343906 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:58.322305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:58.331882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049c124-815d-4827-8949-2daaaefa5d23 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:58.344316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:58.352785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409df2d-5e86-430b-9904-dc63efb00221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:58.353905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409df2d-5e86-430b-9904-dc63efb00221 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:58.354203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409df2d-5e86-430b-9904-dc63efb00221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:58.354487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409df2d-5e86-430b-9904-dc63efb00221 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:58.355496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409df2d-5e86-430b-9904-dc63efb00221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:58.357578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409df2d-5e86-430b-9904-dc63efb00221 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:58.402861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b558e-4ede-4a48-820d-dd934241d32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:58.403738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b558e-4ede-4a48-820d-dd934241d32e admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:58.404249 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b558e-4ede-4a48-820d-dd934241d32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:58.404681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b558e-4ede-4a48-820d-dd934241d32e admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:58.405563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b558e-4ede-4a48-820d-dd934241d32e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:58.408560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b558e-4ede-4a48-820d-dd934241d32e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:58.422724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 914/1790] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:58 2022] POST /identity/v3/users => generated 310 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:22:58.431708 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409df2d-5e86-430b-9904-dc63efb00221 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:58.472539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409df2d-5e86-430b-9904-dc63efb00221 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:58.487143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f01df-c03a-4911-8a21-0ed88c975140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:58.488909 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 877/1791] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:22:57 2022] POST /identity/v3/projects => generated 381 bytes in 824 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:58.498021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b558e-4ede-4a48-820d-dd934241d32e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:58.511491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f01df-c03a-4911-8a21-0ed88c975140 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:58.512922 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f01df-c03a-4911-8a21-0ed88c975140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:58.517436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f01df-c03a-4911-8a21-0ed88c975140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:58.523254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f01df-c03a-4911-8a21-0ed88c975140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:58.527280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f01df-c03a-4911-8a21-0ed88c975140 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:58.550325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 915/1792] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:58.555350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f01df-c03a-4911-8a21-0ed88c975140 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:58.582558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f01df-c03a-4911-8a21-0ed88c975140 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:58.654995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b558e-4ede-4a48-820d-dd934241d32e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:58.679865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df6ec9-b48b-47be-80cc-2feb29f1fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:58.680661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df6ec9-b48b-47be-80cc-2feb29f1fbc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:58.681490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df6ec9-b48b-47be-80cc-2feb29f1fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:58.682030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df6ec9-b48b-47be-80cc-2feb29f1fbc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:58.682940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df6ec9-b48b-47be-80cc-2feb29f1fbc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:58.685342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df6ec9-b48b-47be-80cc-2feb29f1fbc0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:58.741079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 878/1793] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:22:58 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:22:58.745190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df6ec9-b48b-47be-80cc-2feb29f1fbc0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:58.772528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df6ec9-b48b-47be-80cc-2feb29f1fbc0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:58.791519 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c53fd-ef3b-40b4-98d8-1cf800f8b555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:58.793913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c53fd-ef3b-40b4-98d8-1cf800f8b555 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:58.794839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c53fd-ef3b-40b4-98d8-1cf800f8b555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:58.795656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c53fd-ef3b-40b4-98d8-1cf800f8b555 admin admin] PATH_INFO: `/v3/projects/a430575388be42689dfb33d53cb0cc13/users/73977ec8a372460aa119820c7b8a9465/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:58.798825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ee7bf-b957-4c93-abe6-39d78a81d409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:58.799509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ee7bf-b957-4c93-abe6-39d78a81d409 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:58.799831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ee7bf-b957-4c93-abe6-39d78a81d409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:58.800099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ee7bf-b957-4c93-abe6-39d78a81d409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:58.800875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ee7bf-b957-4c93-abe6-39d78a81d409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:58.802911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ee7bf-b957-4c93-abe6-39d78a81d409 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:58.883578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 879/1794] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:58.888025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ee7bf-b957-4c93-abe6-39d78a81d409 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:58.892380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c53fd-ef3b-40b4-98d8-1cf800f8b555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a430575388be42689dfb33d53cb0cc13, user_id=73977ec8a372460aa119820c7b8a9465, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:58.903346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c53fd-ef3b-40b4-98d8-1cf800f8b555 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:58.927326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ee7bf-b957-4c93-abe6-39d78a81d409 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:58.938368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 916/1795] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:58 2022] PUT /identity/v3/projects/a430575388be42689dfb33d53cb0cc13/users/73977ec8a372460aa119820c7b8a9465/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:58.942328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c53fd-ef3b-40b4-98d8-1cf800f8b555 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:59.171948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d685f2-04ad-4b1b-bd6c-777359af6e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:59.173266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d685f2-04ad-4b1b-bd6c-777359af6e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:59.175914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c53fd-ef3b-40b4-98d8-1cf800f8b555 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:59.176995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d685f2-04ad-4b1b-bd6c-777359af6e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:59.177564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d685f2-04ad-4b1b-bd6c-777359af6e8e admin admin] PATH_INFO: `/v3/projects/f1288c76e4dd40cfbe65a43afe74d789/users/3164a70c28f84885a38843fafdbb69f1/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:59.199044 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55d72e-38c3-4f24-b404-228738b23029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:59.200238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55d72e-38c3-4f24-b404-228738b23029 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:59.200753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55d72e-38c3-4f24-b404-228738b23029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:59.201263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55d72e-38c3-4f24-b404-228738b23029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:59.202864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55d72e-38c3-4f24-b404-228738b23029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:59.208131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55d72e-38c3-4f24-b404-228738b23029 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:59.234341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d685f2-04ad-4b1b-bd6c-777359af6e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1288c76e4dd40cfbe65a43afe74d789, user_id=3164a70c28f84885a38843fafdbb69f1, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:59.237103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d685f2-04ad-4b1b-bd6c-777359af6e8e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:59.249139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 917/1796] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:22:59.256849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55d72e-38c3-4f24-b404-228738b23029 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:59.271905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 880/1797] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:58 2022] PUT /identity/v3/projects/f1288c76e4dd40cfbe65a43afe74d789/users/3164a70c28f84885a38843fafdbb69f1/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:59.279614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d685f2-04ad-4b1b-bd6c-777359af6e8e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:59.286834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55d72e-38c3-4f24-b404-228738b23029 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:59.369809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:59.451319 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d685f2-04ad-4b1b-bd6c-777359af6e8e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:59.460727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6a396-c2a4-41e8-9320-3bd3d19c87de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:59.461975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6a396-c2a4-41e8-9320-3bd3d19c87de admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:59.462480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6a396-c2a4-41e8-9320-3bd3d19c87de admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:59.463357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6a396-c2a4-41e8-9320-3bd3d19c87de admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:59.464473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6a396-c2a4-41e8-9320-3bd3d19c87de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:59.467145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6a396-c2a4-41e8-9320-3bd3d19c87de admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:59.467589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:22:59.488360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 881/1798] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:22:59.496323 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6a396-c2a4-41e8-9320-3bd3d19c87de admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:59.511137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6a396-c2a4-41e8-9320-3bd3d19c87de admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:22:59.514068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b0fb4-6c3b-4de3-8df7-e635b5170ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:59.516380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b0fb4-6c3b-4de3-8df7-e635b5170ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:59.516963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b0fb4-6c3b-4de3-8df7-e635b5170ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:59.517536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b0fb4-6c3b-4de3-8df7-e635b5170ee1 admin admin] PATH_INFO: `/v3/projects/a430575388be42689dfb33d53cb0cc13/users/73977ec8a372460aa119820c7b8a9465/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:59.519616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e604a-ba94-4263-9385-131de46fddd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:22:59.521171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e604a-ba94-4263-9385-131de46fddd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:59.521617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e604a-ba94-4263-9385-131de46fddd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:59.522091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e604a-ba94-4263-9385-131de46fddd5 admin admin] PATH_INFO: `/v3/projects/f1288c76e4dd40cfbe65a43afe74d789/users/3164a70c28f84885a38843fafdbb69f1/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:59.523534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b0fb4-6c3b-4de3-8df7-e635b5170ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a430575388be42689dfb33d53cb0cc13, user_id=73977ec8a372460aa119820c7b8a9465, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:59.525769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e604a-ba94-4263-9385-131de46fddd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1288c76e4dd40cfbe65a43afe74d789, user_id=3164a70c28f84885a38843fafdbb69f1, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:22:59.529646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e604a-ba94-4263-9385-131de46fddd5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:59.534075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b0fb4-6c3b-4de3-8df7-e635b5170ee1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:22:59.547009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 882/1799] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:59 2022] PUT /identity/v3/projects/f1288c76e4dd40cfbe65a43afe74d789/users/3164a70c28f84885a38843fafdbb69f1/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:59.564609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 918/1800] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:22:59 2022] PUT /identity/v3/projects/a430575388be42689dfb33d53cb0cc13/users/73977ec8a372460aa119820c7b8a9465/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:22:59.570145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002b69a-e058-4516-ae46-59558814c656 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:59.570786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002b69a-e058-4516-ae46-59558814c656 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:59.574662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002b69a-e058-4516-ae46-59558814c656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:59.576674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc75639-04c3-4b87-8c75-d4288cfb149b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:22:59.577308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc75639-04c3-4b87-8c75-d4288cfb149b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:22:59.577811 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc75639-04c3-4b87-8c75-d4288cfb149b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:22:59.645618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a002b69a-e058-4516-ae46-59558814c656 None None] MFA Rules not processed for user `3164a70c28f84885a38843fafdbb69f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:59.647133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a002b69a-e058-4516-ae46-59558814c656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:59.685754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9dc75639-04c3-4b87-8c75-d4288cfb149b None None] MFA Rules not processed for user `73977ec8a372460aa119820c7b8a9465`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:22:59.687311 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9dc75639-04c3-4b87-8c75-d4288cfb149b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:59.887538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 883/1801] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:59.903581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 919/1802] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:22:59.923453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002b69a-e058-4516-ae46-59558814c656 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:59.927225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc75639-04c3-4b87-8c75-d4288cfb149b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:22:59.929234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9dc75639-04c3-4b87-8c75-d4288cfb149b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:22:59.935580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a002b69a-e058-4516-ae46-59558814c656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:00.123336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc75639-04c3-4b87-8c75-d4288cfb149b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:00.140211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c17bd-993f-43c0-a52e-833edc2ee187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:00.140909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c17bd-993f-43c0-a52e-833edc2ee187 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:00.141192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c17bd-993f-43c0-a52e-833edc2ee187 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:00.141481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c17bd-993f-43c0-a52e-833edc2ee187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:00.170080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c17bd-993f-43c0-a52e-833edc2ee187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:00.173242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c17bd-993f-43c0-a52e-833edc2ee187 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:00.218331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 920/1803] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:59 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:23:00.276055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002b69a-e058-4516-ae46-59558814c656 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:00.296306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00812351-0531-4d47-b739-5d61bf73938d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:00.301037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00812351-0531-4d47-b739-5d61bf73938d service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:00.302022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00812351-0531-4d47-b739-5d61bf73938d service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:00.302986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00812351-0531-4d47-b739-5d61bf73938d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:00.328399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00812351-0531-4d47-b739-5d61bf73938d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:00.335678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00812351-0531-4d47-b739-5d61bf73938d service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:00.407039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 884/1804] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:22:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:00.709667 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c17bd-993f-43c0-a52e-833edc2ee187 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:00.739104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c17bd-993f-43c0-a52e-833edc2ee187 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:00.762387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba328941-6b45-46e7-a309-18b017679e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:00.766062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba328941-6b45-46e7-a309-18b017679e95 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:00.767270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba328941-6b45-46e7-a309-18b017679e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:00.767270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba328941-6b45-46e7-a309-18b017679e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:00.951184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba328941-6b45-46e7-a309-18b017679e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:00.953233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba328941-6b45-46e7-a309-18b017679e95 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:00.979197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 921/1805] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:01.905004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00812351-0531-4d47-b739-5d61bf73938d service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:02.003298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:02.036965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00812351-0531-4d47-b739-5d61bf73938d service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:02.043548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:02.050175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:02.050175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:02.050175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] PATH_INFO: `/v3/users/0465889fa6144da09f9dca0fad2f15b8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:02.053512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0465889fa6144da09f9dca0fad2f15b8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:02.055822 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:02.178685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0465889fa6144da09f9dca0fad2f15b8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:02.205745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0465889fa6144da09f9dca0fad2f15b8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:02.207085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0465889fa6144da09f9dca0fad2f15b8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:02.264132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0465889fa6144da09f9dca0fad2f15b8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:02.270876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0465889fa6144da09f9dca0fad2f15b8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:02.359608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 885/1806] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:01 2022] DELETE /identity/v3/users/0465889fa6144da09f9dca0fad2f15b8 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:02.366461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9078b-f44b-4145-b3a1-0507ada8ca6d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:02.367031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9078b-f44b-4145-b3a1-0507ada8ca6d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:02.367031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9078b-f44b-4145-b3a1-0507ada8ca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:02.418649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-09c9078b-f44b-4145-b3a1-0507ada8ca6d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:02.425997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-09c9078b-f44b-4145-b3a1-0507ada8ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:02.447450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:02.557884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 922/1807] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:02.640964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:02.643668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:02.806172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134850b0-7781-4481-b1ee-aa2519e9dc54 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:02.812625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb85a2-15c2-46b1-aba4-af1668e5695b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:02.813811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb85a2-15c2-46b1-aba4-af1668e5695b service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:02.814993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb85a2-15c2-46b1-aba4-af1668e5695b service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:02.816188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb85a2-15c2-46b1-aba4-af1668e5695b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:02.843267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb85a2-15c2-46b1-aba4-af1668e5695b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:02.847009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb85a2-15c2-46b1-aba4-af1668e5695b service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:02.870117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 886/1808] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:23:03.365516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed3f0b-1dc4-4758-83ef-0edbcc265338 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:03.367581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed3f0b-1dc4-4758-83ef-0edbcc265338 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:03.371286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed3f0b-1dc4-4758-83ef-0edbcc265338 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:03.389369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 923/1809] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:23:03 2022] GET /identity => generated 270 bytes in 29 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:23:03.451140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91a744-51e3-481b-9ffb-ccac39e4bb92 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:03.454496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91a744-51e3-481b-9ffb-ccac39e4bb92 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:03.455622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91a744-51e3-481b-9ffb-ccac39e4bb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:03.578079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fd91a744-51e3-481b-9ffb-ccac39e4bb92 None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:03.580421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fd91a744-51e3-481b-9ffb-ccac39e4bb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:03.683998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:03.710572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 887/1810] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:23:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:03.851281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed3f0b-1dc4-4758-83ef-0edbcc265338 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:03.856489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-71ed3f0b-1dc4-4758-83ef-0edbcc265338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:03.881495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed3f0b-1dc4-4758-83ef-0edbcc265338 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:03.889206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83443ddf-f222-45e2-8d99-4f9d7d9d5c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:03.890467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83443ddf-f222-45e2-8d99-4f9d7d9d5c64 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:03.890977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83443ddf-f222-45e2-8d99-4f9d7d9d5c64 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:03.891457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83443ddf-f222-45e2-8d99-4f9d7d9d5c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:03.904566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83443ddf-f222-45e2-8d99-4f9d7d9d5c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:03.906746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83443ddf-f222-45e2-8d99-4f9d7d9d5c64 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:03.923386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 924/1811] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:04.418495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2d79-af58-4165-ae10-40d9e85822d6 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:04.419123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2d79-af58-4165-ae10-40d9e85822d6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:04.419764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2d79-af58-4165-ae10-40d9e85822d6 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:04.421945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 888/1812] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:23:04.528042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4957c-5d80-48e6-867d-5217a4c168e0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:04.529341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4957c-5d80-48e6-867d-5217a4c168e0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:04.529889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4957c-5d80-48e6-867d-5217a4c168e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:04.578082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-68a4957c-5d80-48e6-867d-5217a4c168e0 None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:04.582324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-68a4957c-5d80-48e6-867d-5217a4c168e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:04.620789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 925/1813] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:23:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:04.796276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2d79-af58-4165-ae10-40d9e85822d6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:04.836701 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2d79-af58-4165-ae10-40d9e85822d6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:04.851276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ca560-bd53-4011-99e4-fc2828e56eb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:04.857196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ca560-bd53-4011-99e4-fc2828e56eb0 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:04.858186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ca560-bd53-4011-99e4-fc2828e56eb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:04.858916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ca560-bd53-4011-99e4-fc2828e56eb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:04.878289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ca560-bd53-4011-99e4-fc2828e56eb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:04.882738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ca560-bd53-4011-99e4-fc2828e56eb0 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:04.902756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 889/1814] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:05.206753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4957c-5d80-48e6-867d-5217a4c168e0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:05.209448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-68a4957c-5d80-48e6-867d-5217a4c168e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:05.244375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4957c-5d80-48e6-867d-5217a4c168e0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:05.252279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:05.255174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:05.255712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:05.256186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] PATH_INFO: `/v3/projects/a230399e40354b4896cf4e1dd1b3632c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:05.260472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a230399e40354b4896cf4e1dd1b3632c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:05.264187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:05.375712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a230399e40354b4896cf4e1dd1b3632c', 'request_id': 'req-3f1277f3-8747-4b11-8325-c82ed7f7dffe'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.395781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a230399e40354b4896cf4e1dd1b3632c', 'request_id': 'req-3f1277f3-8747-4b11-8325-c82ed7f7dffe'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.406795 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a230399e40354b4896cf4e1dd1b3632c', 'request_id': 'req-3f1277f3-8747-4b11-8325-c82ed7f7dffe'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.429052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] The token cache is being invalidate because project a230399e40354b4896cf4e1dd1b3632c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:05.429650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.432417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.441599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 926/1815] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:05 2022] DELETE /identity/v3/projects/a230399e40354b4896cf4e1dd1b3632c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:05.443586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ca560-bd53-4011-99e4-fc2828e56eb0 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:05.461363 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ca560-bd53-4011-99e4-fc2828e56eb0 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:05.664927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:05.667088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:05.667399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:05.668444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] PATH_INFO: `/v3/users/3164a70c28f84885a38843fafdbb69f1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:05.670427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3164a70c28f84885a38843fafdbb69f1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:05.672551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:05.783796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3164a70c28f84885a38843fafdbb69f1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.800056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3164a70c28f84885a38843fafdbb69f1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.801811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3164a70c28f84885a38843fafdbb69f1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.821304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3164a70c28f84885a38843fafdbb69f1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.822664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c270fd07-2873-45a8-a500-77d01855f810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3164a70c28f84885a38843fafdbb69f1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:05.854120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 890/1816] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:05 2022] DELETE /identity/v3/users/3164a70c28f84885a38843fafdbb69f1 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:06.388375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:06.390208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:06.405907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1277f3-8747-4b11-8325-c82ed7f7dffe admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:06.625817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:06.628572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:06.629176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:06.629666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] PATH_INFO: `/v3/projects/f1288c76e4dd40cfbe65a43afe74d789` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:06.632580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1288c76e4dd40cfbe65a43afe74d789)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:06.638355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:06.706022 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:06.808374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1288c76e4dd40cfbe65a43afe74d789', 'request_id': 'req-5d248c73-4a06-46cd-9ae2-76f1402692f6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:06.826783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1288c76e4dd40cfbe65a43afe74d789', 'request_id': 'req-5d248c73-4a06-46cd-9ae2-76f1402692f6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:06.843895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1288c76e4dd40cfbe65a43afe74d789', 'request_id': 'req-5d248c73-4a06-46cd-9ae2-76f1402692f6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:06.867684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] The token cache is being invalidate because project f1288c76e4dd40cfbe65a43afe74d789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:06.868723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:06.870446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d248c73-4a06-46cd-9ae2-76f1402692f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:06.874129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 927/1817] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:06 2022] DELETE /identity/v3/projects/f1288c76e4dd40cfbe65a43afe74d789 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:06.882522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bda3f4-8191-4979-adc2-7b8eae3ec99f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:06.884750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bda3f4-8191-4979-adc2-7b8eae3ec99f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:06.885685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bda3f4-8191-4979-adc2-7b8eae3ec99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:06.945548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d5bda3f4-8191-4979-adc2-7b8eae3ec99f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:06.948127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d5bda3f4-8191-4979-adc2-7b8eae3ec99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:07.088579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:07.105761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 891/1818] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:07.118685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd454e-c29d-4b6b-a5e4-f854dfefe6e1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:07.119202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd454e-c29d-4b6b-a5e4-f854dfefe6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:07.119795 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd454e-c29d-4b6b-a5e4-f854dfefe6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:07.159633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c2cd454e-c29d-4b6b-a5e4-f854dfefe6e1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:07.163081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c2cd454e-c29d-4b6b-a5e4-f854dfefe6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:07.184247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 928/1819] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:07.201920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bda3f4-8191-4979-adc2-7b8eae3ec99f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:07.232108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bda3f4-8191-4979-adc2-7b8eae3ec99f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:07.243923 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850d0cf-a8a9-44d5-bcc0-ad6cc32717a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:07.245581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850d0cf-a8a9-44d5-bcc0-ad6cc32717a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:07.246494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850d0cf-a8a9-44d5-bcc0-ad6cc32717a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:07.247101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850d0cf-a8a9-44d5-bcc0-ad6cc32717a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:07.248634 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850d0cf-a8a9-44d5-bcc0-ad6cc32717a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:23:07.249281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850d0cf-a8a9-44d5-bcc0-ad6cc32717a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:07.253780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850d0cf-a8a9-44d5-bcc0-ad6cc32717a5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:07.293811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 892/1820] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:23:07.294970 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd454e-c29d-4b6b-a5e4-f854dfefe6e1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:07.332696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd454e-c29d-4b6b-a5e4-f854dfefe6e1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:07.348753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac83b09-87fd-486c-9162-1b4e9bae585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:07.350398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac83b09-87fd-486c-9162-1b4e9bae585d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:07.351138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac83b09-87fd-486c-9162-1b4e9bae585d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:07.353266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac83b09-87fd-486c-9162-1b4e9bae585d admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:07.356543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac83b09-87fd-486c-9162-1b4e9bae585d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:07.361089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac83b09-87fd-486c-9162-1b4e9bae585d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:07.426324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 929/1821] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:07 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:07.433441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850d0cf-a8a9-44d5-bcc0-ad6cc32717a5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:07.456850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850d0cf-a8a9-44d5-bcc0-ad6cc32717a5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:07.578466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb03f7-cfe2-45a3-b81e-c82e6a7a60a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:07.579394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb03f7-cfe2-45a3-b81e-c82e6a7a60a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:07.579751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb03f7-cfe2-45a3-b81e-c82e6a7a60a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:07.580132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb03f7-cfe2-45a3-b81e-c82e6a7a60a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:07.581032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb03f7-cfe2-45a3-b81e-c82e6a7a60a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:07.583429 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb03f7-cfe2-45a3-b81e-c82e6a7a60a0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:07.613782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 893/1822] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:07 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:07.620015 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac83b09-87fd-486c-9162-1b4e9bae585d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:07.647304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac83b09-87fd-486c-9162-1b4e9bae585d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:07.661954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1d30d-34f9-46ac-a8af-711989dceb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:07.663459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1d30d-34f9-46ac-a8af-711989dceb0e admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:07.664114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1d30d-34f9-46ac-a8af-711989dceb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:07.665197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1d30d-34f9-46ac-a8af-711989dceb0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:07.666301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1d30d-34f9-46ac-a8af-711989dceb0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:07.668789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1d30d-34f9-46ac-a8af-711989dceb0e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:07.693468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 930/1823] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:07.695655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb03f7-cfe2-45a3-b81e-c82e6a7a60a0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:07.722249 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb03f7-cfe2-45a3-b81e-c82e6a7a60a0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:07.735081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3137de-521a-4891-84dc-a68715178573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:07.736254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3137de-521a-4891-84dc-a68715178573 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:07.736751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3137de-521a-4891-84dc-a68715178573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:07.737231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3137de-521a-4891-84dc-a68715178573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:07.739235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3137de-521a-4891-84dc-a68715178573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:07.743889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3137de-521a-4891-84dc-a68715178573 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:07.770216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 894/1824] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:23:07.771563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1d30d-34f9-46ac-a8af-711989dceb0e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:07.785581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1d30d-34f9-46ac-a8af-711989dceb0e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:07.791087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63696cb8-8f76-4062-bcf5-02ae415f6a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:07.792220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63696cb8-8f76-4062-bcf5-02ae415f6a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:07.792929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63696cb8-8f76-4062-bcf5-02ae415f6a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:07.793330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63696cb8-8f76-4062-bcf5-02ae415f6a7b admin admin] PATH_INFO: `/v3/projects/d27d9750c66e41e182d030f53518b8b5/users/985658dbdb7c41a59dc296b15d85e5db/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:07.822314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63696cb8-8f76-4062-bcf5-02ae415f6a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27d9750c66e41e182d030f53518b8b5, user_id=985658dbdb7c41a59dc296b15d85e5db, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:07.829583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63696cb8-8f76-4062-bcf5-02ae415f6a7b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:07.865332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 931/1825] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:07 2022] PUT /identity/v3/projects/d27d9750c66e41e182d030f53518b8b5/users/985658dbdb7c41a59dc296b15d85e5db/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:07.876882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d257-7e8a-4c50-8d64-2d44babc1670 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:07.877184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d257-7e8a-4c50-8d64-2d44babc1670 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:07.877466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d257-7e8a-4c50-8d64-2d44babc1670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:07.913225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4395d257-7e8a-4c50-8d64-2d44babc1670 None None] MFA Rules not processed for user `985658dbdb7c41a59dc296b15d85e5db`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:07.916094 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4395d257-7e8a-4c50-8d64-2d44babc1670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:08.037254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 895/1826] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:08.038672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63696cb8-8f76-4062-bcf5-02ae415f6a7b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:08.253188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63696cb8-8f76-4062-bcf5-02ae415f6a7b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:08.260414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bafe2-594d-4783-bfc0-97fc1701c4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:08.261373 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bafe2-594d-4783-bfc0-97fc1701c4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:08.261769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bafe2-594d-4783-bfc0-97fc1701c4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:08.262129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bafe2-594d-4783-bfc0-97fc1701c4a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:08.263111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73bafe2-594d-4783-bfc0-97fc1701c4a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:08.266980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73bafe2-594d-4783-bfc0-97fc1701c4a5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:08.332936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 932/1827] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:08 2022] POST /identity/v3/projects => generated 385 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:08.335628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395d257-7e8a-4c50-8d64-2d44babc1670 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:08.356589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395d257-7e8a-4c50-8d64-2d44babc1670 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:08.462360 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:08.478746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e729aee-25a5-4b46-955a-0117171531e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:08.480396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e729aee-25a5-4b46-955a-0117171531e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:08.481486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e729aee-25a5-4b46-955a-0117171531e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:08.481682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e729aee-25a5-4b46-955a-0117171531e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:08.482574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e729aee-25a5-4b46-955a-0117171531e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:08.488324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e729aee-25a5-4b46-955a-0117171531e1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:08.525472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 896/1828] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:08 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:08.527503 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bafe2-594d-4783-bfc0-97fc1701c4a5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:08.551513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bafe2-594d-4783-bfc0-97fc1701c4a5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:08.566374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77837cab-8860-4370-8b7c-f719842414a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:08.567595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77837cab-8860-4370-8b7c-f719842414a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:08.568114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77837cab-8860-4370-8b7c-f719842414a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:08.568587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77837cab-8860-4370-8b7c-f719842414a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:08.573176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77837cab-8860-4370-8b7c-f719842414a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:08.582362 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77837cab-8860-4370-8b7c-f719842414a7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:08.618164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 933/1829] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:08.625234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e729aee-25a5-4b46-955a-0117171531e1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:08.647137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e729aee-25a5-4b46-955a-0117171531e1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:08.662724 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a39023-f1a6-43bd-9408-05893c1bfd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:08.665448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a39023-f1a6-43bd-9408-05893c1bfd1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:08.666054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a39023-f1a6-43bd-9408-05893c1bfd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:08.666458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a39023-f1a6-43bd-9408-05893c1bfd1f admin admin] PATH_INFO: `/v3/projects/d42087027f5b4f43a96b61a01a960231/users/bda95a0b3a48432f9be6ee8310437c7f/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:08.710193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a39023-f1a6-43bd-9408-05893c1bfd1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42087027f5b4f43a96b61a01a960231, user_id=bda95a0b3a48432f9be6ee8310437c7f, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:08.719711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a39023-f1a6-43bd-9408-05893c1bfd1f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:08.745809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 897/1830] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:08 2022] PUT /identity/v3/projects/d42087027f5b4f43a96b61a01a960231/users/bda95a0b3a48432f9be6ee8310437c7f/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:08.751015 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77837cab-8860-4370-8b7c-f719842414a7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:08.781259 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77837cab-8860-4370-8b7c-f719842414a7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:08.837886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:08.939716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686721b-a1af-4360-b89b-c1a3db7882a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:08.941202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686721b-a1af-4360-b89b-c1a3db7882a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:08.941888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686721b-a1af-4360-b89b-c1a3db7882a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:08.942649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686721b-a1af-4360-b89b-c1a3db7882a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:08.944788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686721b-a1af-4360-b89b-c1a3db7882a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:08.948470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686721b-a1af-4360-b89b-c1a3db7882a5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:08.980521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 934/1831] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:08.987055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a39023-f1a6-43bd-9408-05893c1bfd1f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:09.004456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a39023-f1a6-43bd-9408-05893c1bfd1f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:09.033900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09899557-6534-48a6-ada2-3f2af5bd628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:09.037170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09899557-6534-48a6-ada2-3f2af5bd628d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:09.038046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09899557-6534-48a6-ada2-3f2af5bd628d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:09.038809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09899557-6534-48a6-ada2-3f2af5bd628d admin admin] PATH_INFO: `/v3/projects/d42087027f5b4f43a96b61a01a960231/users/bda95a0b3a48432f9be6ee8310437c7f/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:09.044529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09899557-6534-48a6-ada2-3f2af5bd628d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42087027f5b4f43a96b61a01a960231, user_id=bda95a0b3a48432f9be6ee8310437c7f, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:09.051230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09899557-6534-48a6-ada2-3f2af5bd628d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:09.083720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 898/1832] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:08 2022] PUT /identity/v3/projects/d42087027f5b4f43a96b61a01a960231/users/bda95a0b3a48432f9be6ee8310437c7f/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:09.095169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6254-0e71-48a3-95af-0bf244345341 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:09.095510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6254-0e71-48a3-95af-0bf244345341 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:09.095725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6254-0e71-48a3-95af-0bf244345341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:09.156748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c4ad6254-0e71-48a3-95af-0bf244345341 None None] MFA Rules not processed for user `bda95a0b3a48432f9be6ee8310437c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:09.158041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c4ad6254-0e71-48a3-95af-0bf244345341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:09.341832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 935/1833] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:09.368172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09899557-6534-48a6-ada2-3f2af5bd628d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:09.372688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-09899557-6534-48a6-ada2-3f2af5bd628d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:09.577882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09899557-6534-48a6-ada2-3f2af5bd628d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:09.589673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90abb90-8882-4626-9dc9-d10e1475d475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:09.590744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90abb90-8882-4626-9dc9-d10e1475d475 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:09.591159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90abb90-8882-4626-9dc9-d10e1475d475 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:09.591634 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90abb90-8882-4626-9dc9-d10e1475d475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:09.822794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90abb90-8882-4626-9dc9-d10e1475d475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:09.826191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90abb90-8882-4626-9dc9-d10e1475d475 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:09.869461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 899/1834] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:12.348797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad6254-0e71-48a3-95af-0bf244345341 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:12.352065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c4ad6254-0e71-48a3-95af-0bf244345341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:12.525001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad6254-0e71-48a3-95af-0bf244345341 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:12.533253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:12.535203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:12.535525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:12.535843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] PATH_INFO: `/v3/users/86a8b8b6e0da4cd4823cb7c8fcd48d84` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:12.538420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a8b8b6e0da4cd4823cb7c8fcd48d84)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:12.541113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:12.666594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:12.685982 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a8b8b6e0da4cd4823cb7c8fcd48d84'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:12.710487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a8b8b6e0da4cd4823cb7c8fcd48d84'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:12.713967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a8b8b6e0da4cd4823cb7c8fcd48d84'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:12.734217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a8b8b6e0da4cd4823cb7c8fcd48d84'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:12.770075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a8b8b6e0da4cd4823cb7c8fcd48d84'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:12.793827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 936/1835] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:12 2022] DELETE /identity/v3/users/86a8b8b6e0da4cd4823cb7c8fcd48d84 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:12.803807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb0149-51fb-4406-b0b6-58ecee7d41b7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:12.806900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb0149-51fb-4406-b0b6-58ecee7d41b7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:12.811853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb0149-51fb-4406-b0b6-58ecee7d41b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:12.874430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b6fb0149-51fb-4406-b0b6-58ecee7d41b7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:12.880545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b6fb0149-51fb-4406-b0b6-58ecee7d41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:12.964064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 900/1836] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:12.975197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:12.977775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:13.057292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebae51-a526-4161-903f-0a20a07ec7dc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:13.066344 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3939-9fd1-4a82-a0f4-7bc9aa8cd757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:13.067797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3939-9fd1-4a82-a0f4-7bc9aa8cd757 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:13.068595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3939-9fd1-4a82-a0f4-7bc9aa8cd757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:13.068998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3939-9fd1-4a82-a0f4-7bc9aa8cd757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:13.083202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3939-9fd1-4a82-a0f4-7bc9aa8cd757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:13.084849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3939-9fd1-4a82-a0f4-7bc9aa8cd757 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:13.097871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 937/1837] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:13.571836 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb0149-51fb-4406-b0b6-58ecee7d41b7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:13.574126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b6fb0149-51fb-4406-b0b6-58ecee7d41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:13.596529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:13.597003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb0149-51fb-4406-b0b6-58ecee7d41b7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:13.607053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:13.608902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:13.609509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:13.610210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] PATH_INFO: `/v3/projects/6b6a1ab356ae4dd7828d6010af911a4d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:13.612986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b6a1ab356ae4dd7828d6010af911a4d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:13.617760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:13.748293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b6a1ab356ae4dd7828d6010af911a4d', 'request_id': 'req-55d3d119-9a47-4aae-887f-8694b2155bfc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:13.762300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b6a1ab356ae4dd7828d6010af911a4d', 'request_id': 'req-55d3d119-9a47-4aae-887f-8694b2155bfc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:13.776542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b6a1ab356ae4dd7828d6010af911a4d', 'request_id': 'req-55d3d119-9a47-4aae-887f-8694b2155bfc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:13.808591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] The token cache is being invalidate because project 6b6a1ab356ae4dd7828d6010af911a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:13.809688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:13.810652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:13.812623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 901/1838] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:13 2022] DELETE /identity/v3/projects/6b6a1ab356ae4dd7828d6010af911a4d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:13.818558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3939-9fd1-4a82-a0f4-7bc9aa8cd757 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:13.835394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3939-9fd1-4a82-a0f4-7bc9aa8cd757 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:13.917632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:13.918879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:13.919376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:13.919781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] PATH_INFO: `/v3/users/73977ec8a372460aa119820c7b8a9465` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:13.921568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73977ec8a372460aa119820c7b8a9465)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:13.924566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:14.004374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73977ec8a372460aa119820c7b8a9465'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:14.022639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73977ec8a372460aa119820c7b8a9465'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:14.024119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73977ec8a372460aa119820c7b8a9465'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:14.036168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73977ec8a372460aa119820c7b8a9465'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:14.038927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:14.075672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1074003e-b74e-4a62-9a75-c3d6ab3b9a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73977ec8a372460aa119820c7b8a9465'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:14.091041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 938/1839] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:13 2022] DELETE /identity/v3/users/73977ec8a372460aa119820c7b8a9465 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:14.696025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:14.698583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:14.724149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d119-9a47-4aae-887f-8694b2155bfc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:14.863623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:14.864548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:14.864833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:14.865104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] PATH_INFO: `/v3/projects/a430575388be42689dfb33d53cb0cc13` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:14.866496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a430575388be42689dfb33d53cb0cc13)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:14.869290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:14.951048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a430575388be42689dfb33d53cb0cc13', 'request_id': 'req-68c1222f-2755-4153-8998-447e85e861ab'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:14.966777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a430575388be42689dfb33d53cb0cc13', 'request_id': 'req-68c1222f-2755-4153-8998-447e85e861ab'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:14.982558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a430575388be42689dfb33d53cb0cc13', 'request_id': 'req-68c1222f-2755-4153-8998-447e85e861ab'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:15.006053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] The token cache is being invalidate because project a430575388be42689dfb33d53cb0cc13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:15.006690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:15.007799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-68c1222f-2755-4153-8998-447e85e861ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:15.014592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 902/1840] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:14 2022] DELETE /identity/v3/projects/a430575388be42689dfb33d53cb0cc13 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:15.021771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f4f95-5e14-413c-831c-cc2c1dcb1a36 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:15.022289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f4f95-5e14-413c-831c-cc2c1dcb1a36 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:15.022391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f4f95-5e14-413c-831c-cc2c1dcb1a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:15.065289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ba5f4f95-5e14-413c-831c-cc2c1dcb1a36 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:15.067301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ba5f4f95-5e14-413c-831c-cc2c1dcb1a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:15.192419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 939/1841] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:15.201673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aeaa42-be2f-410c-9bae-6e51475ef3bc None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:15.202273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aeaa42-be2f-410c-9bae-6e51475ef3bc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:15.202831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aeaa42-be2f-410c-9bae-6e51475ef3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:15.240991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-72aeaa42-be2f-410c-9bae-6e51475ef3bc None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:15.244142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-72aeaa42-be2f-410c-9bae-6e51475ef3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:15.267829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 903/1842] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:15.272497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f4f95-5e14-413c-831c-cc2c1dcb1a36 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:15.294080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f4f95-5e14-413c-831c-cc2c1dcb1a36 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:15.311655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceab4bd-bc8f-417f-b0aa-0e5bf8c42bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:15.312284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceab4bd-bc8f-417f-b0aa-0e5bf8c42bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:15.312782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceab4bd-bc8f-417f-b0aa-0e5bf8c42bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:15.313610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceab4bd-bc8f-417f-b0aa-0e5bf8c42bbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:15.315005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceab4bd-bc8f-417f-b0aa-0e5bf8c42bbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:23:15.315569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceab4bd-bc8f-417f-b0aa-0e5bf8c42bbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:15.318812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceab4bd-bc8f-417f-b0aa-0e5bf8c42bbe admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:15.343354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 940/1843] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:23:15.348976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aeaa42-be2f-410c-9bae-6e51475ef3bc None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:15.365880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aeaa42-be2f-410c-9bae-6e51475ef3bc None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:15.372681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06eb7d8-e8a9-41f4-8ca3-14594b6058bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:15.373593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eb7d8-e8a9-41f4-8ca3-14594b6058bf admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:15.374013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eb7d8-e8a9-41f4-8ca3-14594b6058bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:15.374374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eb7d8-e8a9-41f4-8ca3-14594b6058bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:15.375202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06eb7d8-e8a9-41f4-8ca3-14594b6058bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:15.377530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06eb7d8-e8a9-41f4-8ca3-14594b6058bf admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:15.382875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:15.416237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 904/1844] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:15 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:15.420926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceab4bd-bc8f-417f-b0aa-0e5bf8c42bbe admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:15.435150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceab4bd-bc8f-417f-b0aa-0e5bf8c42bbe admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:15.507132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeac82-48e1-40b9-8114-534b8b6fcd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:15.507755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeac82-48e1-40b9-8114-534b8b6fcd44 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:15.508380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeac82-48e1-40b9-8114-534b8b6fcd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:15.508613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeac82-48e1-40b9-8114-534b8b6fcd44 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:15.509183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaeac82-48e1-40b9-8114-534b8b6fcd44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:15.511259 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaeac82-48e1-40b9-8114-534b8b6fcd44 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:15.536323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 941/1845] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:15 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:15.541598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06eb7d8-e8a9-41f4-8ca3-14594b6058bf admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:15.553200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06eb7d8-e8a9-41f4-8ca3-14594b6058bf admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:15.560005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f83044-7445-493b-90a7-abe9f9be2720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:15.560803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f83044-7445-493b-90a7-abe9f9be2720 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:15.561164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f83044-7445-493b-90a7-abe9f9be2720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:15.561510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f83044-7445-493b-90a7-abe9f9be2720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:15.562412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f83044-7445-493b-90a7-abe9f9be2720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:15.564268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f83044-7445-493b-90a7-abe9f9be2720 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:15.576150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 905/1846] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:23:15.580849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeac82-48e1-40b9-8114-534b8b6fcd44 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:15.593104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeac82-48e1-40b9-8114-534b8b6fcd44 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:15.598408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9af40-807d-4fc4-825d-e6b3e310bef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:15.599102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9af40-807d-4fc4-825d-e6b3e310bef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:15.599392 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9af40-807d-4fc4-825d-e6b3e310bef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:15.599664 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9af40-807d-4fc4-825d-e6b3e310bef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:15.600406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9af40-807d-4fc4-825d-e6b3e310bef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:15.602804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9af40-807d-4fc4-825d-e6b3e310bef8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:15.615516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 942/1847] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:23:15.620593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f83044-7445-493b-90a7-abe9f9be2720 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:15.632651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f83044-7445-493b-90a7-abe9f9be2720 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:15.638603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874efa42-53de-49af-bcb8-ee5aa5ec4225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:15.639519 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874efa42-53de-49af-bcb8-ee5aa5ec4225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:15.639799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874efa42-53de-49af-bcb8-ee5aa5ec4225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:15.640078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874efa42-53de-49af-bcb8-ee5aa5ec4225 admin admin] PATH_INFO: `/v3/projects/9e585669e8ba49e684ca2c0913b8280f/users/1c74aa2746c845eea736dacf06a1a537/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:15.657808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874efa42-53de-49af-bcb8-ee5aa5ec4225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e585669e8ba49e684ca2c0913b8280f, user_id=1c74aa2746c845eea736dacf06a1a537, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:15.660588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874efa42-53de-49af-bcb8-ee5aa5ec4225 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:15.681139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 906/1848] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:15 2022] PUT /identity/v3/projects/9e585669e8ba49e684ca2c0913b8280f/users/1c74aa2746c845eea736dacf06a1a537/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:15.704171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052ad08-2c10-41a6-b65f-197672facd4e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:15.704628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052ad08-2c10-41a6-b65f-197672facd4e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:15.705085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052ad08-2c10-41a6-b65f-197672facd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:15.762149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1052ad08-2c10-41a6-b65f-197672facd4e None None] MFA Rules not processed for user `1c74aa2746c845eea736dacf06a1a537`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:15.764232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1052ad08-2c10-41a6-b65f-197672facd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:15.824776 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:15.877591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 943/1849] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:15.892753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874efa42-53de-49af-bcb8-ee5aa5ec4225 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:15.895206 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-874efa42-53de-49af-bcb8-ee5aa5ec4225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:16.002378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874efa42-53de-49af-bcb8-ee5aa5ec4225 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:16.009566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaab0d-89e9-4e6c-978e-a679d7825579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:16.010406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaab0d-89e9-4e6c-978e-a679d7825579 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:16.010686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaab0d-89e9-4e6c-978e-a679d7825579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:16.010967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaab0d-89e9-4e6c-978e-a679d7825579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:16.022306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaab0d-89e9-4e6c-978e-a679d7825579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:16.024279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaab0d-89e9-4e6c-978e-a679d7825579 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:16.043331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 907/1850] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:16.542125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052ad08-2c10-41a6-b65f-197672facd4e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:16.543660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1052ad08-2c10-41a6-b65f-197672facd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:16.569898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052ad08-2c10-41a6-b65f-197672facd4e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:16.575809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52085b-35f3-48d5-842d-db4a8b5d3f27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:16.576499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52085b-35f3-48d5-842d-db4a8b5d3f27 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:16.576828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52085b-35f3-48d5-842d-db4a8b5d3f27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:16.577144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52085b-35f3-48d5-842d-db4a8b5d3f27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:16.579662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7c52085b-35f3-48d5-842d-db4a8b5d3f27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:16.703642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52085b-35f3-48d5-842d-db4a8b5d3f27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:16.705356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52085b-35f3-48d5-842d-db4a8b5d3f27 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:16.722163 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 944/1851] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:23:25.786358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaab0d-89e9-4e6c-978e-a679d7825579 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:25.790442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0fbaab0d-89e9-4e6c-978e-a679d7825579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:26.082046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaab0d-89e9-4e6c-978e-a679d7825579 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:26.106262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:26.107824 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:26.108301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:26.108784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] PATH_INFO: `/v3/users/9316bbf654cd412684a7f6ca1a8b0002` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:26.115171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9316bbf654cd412684a7f6ca1a8b0002)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:26.123031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:26.284825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9316bbf654cd412684a7f6ca1a8b0002'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.298863 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9316bbf654cd412684a7f6ca1a8b0002'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.301853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9316bbf654cd412684a7f6ca1a8b0002'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.304177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:26.338183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9316bbf654cd412684a7f6ca1a8b0002'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.339116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9316bbf654cd412684a7f6ca1a8b0002'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.368778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 908/1852] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:25 2022] DELETE /identity/v3/users/9316bbf654cd412684a7f6ca1a8b0002 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:26.374630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061731-7c47-414d-8f00-854b8c8117c8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:26.383826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061731-7c47-414d-8f00-854b8c8117c8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:26.384543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061731-7c47-414d-8f00-854b8c8117c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:26.416465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a0061731-7c47-414d-8f00-854b8c8117c8 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:26.417247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a0061731-7c47-414d-8f00-854b8c8117c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:26.556827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 945/1853] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:26.568559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:26.571765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:26.636729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0061731-7c47-414d-8f00-854b8c8117c8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:26.640360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a0061731-7c47-414d-8f00-854b8c8117c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:26.679000 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0061731-7c47-414d-8f00-854b8c8117c8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:26.689772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:26.691880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:26.692307 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:26.692932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] PATH_INFO: `/v3/users/985658dbdb7c41a59dc296b15d85e5db` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:26.697018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=985658dbdb7c41a59dc296b15d85e5db)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:26.701464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:26.778113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fa3c-2b27-44e2-936f-1c88cb2cd727 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:26.783711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa71641-f58f-4a24-a7ca-4d6f7293028d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:26.784602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa71641-f58f-4a24-a7ca-4d6f7293028d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:26.785159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa71641-f58f-4a24-a7ca-4d6f7293028d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:26.785587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa71641-f58f-4a24-a7ca-4d6f7293028d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:26.804105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa71641-f58f-4a24-a7ca-4d6f7293028d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:26.807594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985658dbdb7c41a59dc296b15d85e5db'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.808007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa71641-f58f-4a24-a7ca-4d6f7293028d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:26.825177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 909/1854] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:26.839306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985658dbdb7c41a59dc296b15d85e5db'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.840775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985658dbdb7c41a59dc296b15d85e5db'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.864516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985658dbdb7c41a59dc296b15d85e5db'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.868656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:26.905521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985658dbdb7c41a59dc296b15d85e5db'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:26.928562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 946/1855] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:26 2022] DELETE /identity/v3/users/985658dbdb7c41a59dc296b15d85e5db => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:26.936871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f69003-5017-48e6-886b-fdc8da89e441 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:26.937488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f69003-5017-48e6-886b-fdc8da89e441 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:26.938800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f69003-5017-48e6-886b-fdc8da89e441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:26.975339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-62f69003-5017-48e6-886b-fdc8da89e441 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:26.978364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-62f69003-5017-48e6-886b-fdc8da89e441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:27.121765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 910/1856] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:27.139289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:27.142128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:27.426578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba1509-f922-4759-933d-5d63115e1497 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:27.451294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e156c4-2b2b-4c80-a634-85f5db36f64e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:27.452654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e156c4-2b2b-4c80-a634-85f5db36f64e service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:27.457834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e156c4-2b2b-4c80-a634-85f5db36f64e service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:27.458241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e156c4-2b2b-4c80-a634-85f5db36f64e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:27.513941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e156c4-2b2b-4c80-a634-85f5db36f64e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:27.517039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e156c4-2b2b-4c80-a634-85f5db36f64e service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:27.569191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 947/1857] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:27.890449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f69003-5017-48e6-886b-fdc8da89e441 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:27.893871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-62f69003-5017-48e6-886b-fdc8da89e441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:27.937187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f69003-5017-48e6-886b-fdc8da89e441 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:27.942885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:27.943886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:27.944357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:27.944702 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] PATH_INFO: `/v3/projects/0e0374761b2b46ae84b2f77ab406fab1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:27.947537 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e0374761b2b46ae84b2f77ab406fab1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:27.951918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:28.085746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e0374761b2b46ae84b2f77ab406fab1', 'request_id': 'req-e79206c5-ae15-4115-ae51-32237f139c67'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.098148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e0374761b2b46ae84b2f77ab406fab1', 'request_id': 'req-e79206c5-ae15-4115-ae51-32237f139c67'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.107081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e0374761b2b46ae84b2f77ab406fab1', 'request_id': 'req-e79206c5-ae15-4115-ae51-32237f139c67'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.125907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:28.130507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] The token cache is being invalidate because project 0e0374761b2b46ae84b2f77ab406fab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:28.131342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.133388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.136139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 911/1858] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:27 2022] DELETE /identity/v3/projects/0e0374761b2b46ae84b2f77ab406fab1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:28.140812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e156c4-2b2b-4c80-a634-85f5db36f64e service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:28.159503 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e156c4-2b2b-4c80-a634-85f5db36f64e service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:28.264739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:28.267999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:28.268984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:28.270084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] PATH_INFO: `/v3/users/02d3f706936d4cf58affbe98291b9093` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:28.273350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02d3f706936d4cf58affbe98291b9093)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:28.280422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:28.422512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02d3f706936d4cf58affbe98291b9093'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.440691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02d3f706936d4cf58affbe98291b9093'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.442273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02d3f706936d4cf58affbe98291b9093'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.466343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02d3f706936d4cf58affbe98291b9093'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.524940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02d3f706936d4cf58affbe98291b9093'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:28.545476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 948/1859] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:28 2022] DELETE /identity/v3/users/02d3f706936d4cf58affbe98291b9093 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:29.064737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:29.067026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:29.096238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:29.105585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:29.137444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:29.145902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79206c5-ae15-4115-ae51-32237f139c67 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:29.293893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:29.296598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:29.296868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:29.297180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] PATH_INFO: `/v3/projects/38a16b14fb274f398adc6f322131b5e2` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:29.303386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a16b14fb274f398adc6f322131b5e2)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:29.309663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:29.330874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf544a41-ee01-4ca1-b300-a3c6125947c0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:29.339266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:29.340510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:29.341024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:29.341364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] PATH_INFO: `/v3/projects/d27d9750c66e41e182d030f53518b8b5` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:29.343420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27d9750c66e41e182d030f53518b8b5)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:29.347052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:29.397672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a16b14fb274f398adc6f322131b5e2', 'request_id': 'req-409f24e4-8d4f-495e-bbc2-55329db35f28'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.408465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a16b14fb274f398adc6f322131b5e2', 'request_id': 'req-409f24e4-8d4f-495e-bbc2-55329db35f28'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.419591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a16b14fb274f398adc6f322131b5e2', 'request_id': 'req-409f24e4-8d4f-495e-bbc2-55329db35f28'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.436532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] The token cache is being invalidate because project 38a16b14fb274f398adc6f322131b5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:29.436994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.437741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-409f24e4-8d4f-495e-bbc2-55329db35f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.439841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 912/1860] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:29 2022] DELETE /identity/v3/projects/38a16b14fb274f398adc6f322131b5e2 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:29.446506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae645cc-936d-4c5a-b776-bc01436bcc06 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:29.447086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae645cc-936d-4c5a-b776-bc01436bcc06 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:29.447481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae645cc-936d-4c5a-b776-bc01436bcc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:29.454814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27d9750c66e41e182d030f53518b8b5', 'request_id': 'req-813a007c-7cf2-4055-9749-bd3149b5dcb0'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.470832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27d9750c66e41e182d030f53518b8b5', 'request_id': 'req-813a007c-7cf2-4055-9749-bd3149b5dcb0'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.471221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9ae645cc-936d-4c5a-b776-bc01436bcc06 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:29.472791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9ae645cc-936d-4c5a-b776-bc01436bcc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:29.488460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27d9750c66e41e182d030f53518b8b5', 'request_id': 'req-813a007c-7cf2-4055-9749-bd3149b5dcb0'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.522526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] The token cache is being invalidate because project d27d9750c66e41e182d030f53518b8b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:29.523610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.524815 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.531859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 949/1861] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:29 2022] DELETE /identity/v3/projects/d27d9750c66e41e182d030f53518b8b5 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:29.535257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:29.537637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:29.604376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 913/1862] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:29.624056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874e23e-840d-4ab5-aa48-02dc22013770 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:29.625401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874e23e-840d-4ab5-aa48-02dc22013770 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:29.625832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874e23e-840d-4ab5-aa48-02dc22013770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:29.685346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9874e23e-840d-4ab5-aa48-02dc22013770 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:29.697003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9874e23e-840d-4ab5-aa48-02dc22013770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:29.733987 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 914/1863] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:29.738616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874e23e-840d-4ab5-aa48-02dc22013770 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:29.759871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a007c-7cf2-4055-9749-bd3149b5dcb0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:29.772368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:29.774969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:29.775156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:29.775353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] PATH_INFO: `/v3/users/bda95a0b3a48432f9be6ee8310437c7f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:29.779060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda95a0b3a48432f9be6ee8310437c7f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:29.780476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874e23e-840d-4ab5-aa48-02dc22013770 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:29.786132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:29.838187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d41ffe-3dea-48c9-ad73-4f987ea46fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:29.845022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d41ffe-3dea-48c9-ad73-4f987ea46fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:29.845562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d41ffe-3dea-48c9-ad73-4f987ea46fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:29.846103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d41ffe-3dea-48c9-ad73-4f987ea46fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:29.850258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d41ffe-3dea-48c9-ad73-4f987ea46fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:23:29.857165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d41ffe-3dea-48c9-ad73-4f987ea46fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:29.867113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d41ffe-3dea-48c9-ad73-4f987ea46fe8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:29.896548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:29.920198 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 915/1864] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:23:29.926257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda95a0b3a48432f9be6ee8310437c7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.927549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d41ffe-3dea-48c9-ad73-4f987ea46fe8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:29.947880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda95a0b3a48432f9be6ee8310437c7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.948674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda95a0b3a48432f9be6ee8310437c7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.962612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda95a0b3a48432f9be6ee8310437c7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.983734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda95a0b3a48432f9be6ee8310437c7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:29.987528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d41ffe-3dea-48c9-ad73-4f987ea46fe8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:29.994970 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 950/1865] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:29 2022] DELETE /identity/v3/users/bda95a0b3a48432f9be6ee8310437c7f => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:30.012003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f799662-414e-45e2-a304-790190880b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:30.013515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f799662-414e-45e2-a304-790190880b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:30.016165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f799662-414e-45e2-a304-790190880b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:30.017497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f799662-414e-45e2-a304-790190880b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:30.019232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f799662-414e-45e2-a304-790190880b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:30.023115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f799662-414e-45e2-a304-790190880b5b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:30.053496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:30.123798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 916/1866] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:29 2022] POST /identity/v3/projects => generated 367 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:30.129943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:30.141064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:30.382218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b97616-bb55-46ef-8dcd-a86c193525c6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:30.509804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2829a35-34b1-4c89-80f6-2342af948077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:30.512256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2829a35-34b1-4c89-80f6-2342af948077 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:30.512984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2829a35-34b1-4c89-80f6-2342af948077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:30.513745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2829a35-34b1-4c89-80f6-2342af948077 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:30.515333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2829a35-34b1-4c89-80f6-2342af948077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:30.519588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2829a35-34b1-4c89-80f6-2342af948077 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:30.577387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 951/1867] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:30 2022] POST /identity/v3/users => generated 284 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:30.578753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f799662-414e-45e2-a304-790190880b5b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:30.592575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f799662-414e-45e2-a304-790190880b5b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:30.598298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7aacd-4fd9-4ec8-9618-13b0089a0246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:30.599000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7aacd-4fd9-4ec8-9618-13b0089a0246 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:30.599283 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7aacd-4fd9-4ec8-9618-13b0089a0246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:30.599554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7aacd-4fd9-4ec8-9618-13b0089a0246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:30.600338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7aacd-4fd9-4ec8-9618-13b0089a0246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:30.602579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7aacd-4fd9-4ec8-9618-13b0089a0246 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:30.619734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 917/1868] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:30.622575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2829a35-34b1-4c89-80f6-2342af948077 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:30.636702 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2829a35-34b1-4c89-80f6-2342af948077 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:30.642981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a832e-6c41-4ef1-9d71-c419e4dd51d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:30.643938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a832e-6c41-4ef1-9d71-c419e4dd51d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:30.644347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a832e-6c41-4ef1-9d71-c419e4dd51d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:30.649666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a832e-6c41-4ef1-9d71-c419e4dd51d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:30.652055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a832e-6c41-4ef1-9d71-c419e4dd51d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:30.654000 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a832e-6c41-4ef1-9d71-c419e4dd51d8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:30.677530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 952/1869] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:30.680884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7aacd-4fd9-4ec8-9618-13b0089a0246 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:30.700285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7aacd-4fd9-4ec8-9618-13b0089a0246 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:30.709364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b118397-93bb-41a5-a73e-039a1cdfa48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:30.710993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b118397-93bb-41a5-a73e-039a1cdfa48c admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:30.711472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b118397-93bb-41a5-a73e-039a1cdfa48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:30.713835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b118397-93bb-41a5-a73e-039a1cdfa48c admin admin] PATH_INFO: `/v3/projects/319fc624b55e401085e554c42779ff26/users/74dcc98d05e34302a36bb7e4d78d8208/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:30.746075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b118397-93bb-41a5-a73e-039a1cdfa48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=319fc624b55e401085e554c42779ff26, user_id=74dcc98d05e34302a36bb7e4d78d8208, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:30.748260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b118397-93bb-41a5-a73e-039a1cdfa48c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:30.792705 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 918/1870] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:30 2022] PUT /identity/v3/projects/319fc624b55e401085e554c42779ff26/users/74dcc98d05e34302a36bb7e4d78d8208/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:30.824771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fe46c-1bde-4e30-a121-6a4d9862dc5e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:30.825225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fe46c-1bde-4e30-a121-6a4d9862dc5e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:30.826240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fe46c-1bde-4e30-a121-6a4d9862dc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:30.891258 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a75fe46c-1bde-4e30-a121-6a4d9862dc5e None None] MFA Rules not processed for user `74dcc98d05e34302a36bb7e4d78d8208`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:30.893973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a75fe46c-1bde-4e30-a121-6a4d9862dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:30.960147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:31.031199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 953/1871] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:31.038083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b118397-93bb-41a5-a73e-039a1cdfa48c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:31.227595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b118397-93bb-41a5-a73e-039a1cdfa48c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:31.233397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5c53b-a7b8-44fd-bfd5-1f8653434e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:31.234445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5c53b-a7b8-44fd-bfd5-1f8653434e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:31.234835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5c53b-a7b8-44fd-bfd5-1f8653434e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:31.235394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5c53b-a7b8-44fd-bfd5-1f8653434e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:31.236293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5c53b-a7b8-44fd-bfd5-1f8653434e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:31.238653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5c53b-a7b8-44fd-bfd5-1f8653434e35 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:31.277787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 919/1872] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:31 2022] POST /identity/v3/projects => generated 369 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:31.283334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fe46c-1bde-4e30-a121-6a4d9862dc5e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:31.314137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fe46c-1bde-4e30-a121-6a4d9862dc5e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:31.449971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438aa02-050b-4cb2-bf70-6eba3e48a9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:31.450688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438aa02-050b-4cb2-bf70-6eba3e48a9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:31.450977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438aa02-050b-4cb2-bf70-6eba3e48a9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:31.451368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438aa02-050b-4cb2-bf70-6eba3e48a9f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:31.452080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438aa02-050b-4cb2-bf70-6eba3e48a9f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:31.454183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438aa02-050b-4cb2-bf70-6eba3e48a9f2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:31.479943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 954/1873] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:31 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:31.484513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5c53b-a7b8-44fd-bfd5-1f8653434e35 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:31.497372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5c53b-a7b8-44fd-bfd5-1f8653434e35 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:31.516694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c604035-3f27-4ea0-ad8f-a10edd1374b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:31.518799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c604035-3f27-4ea0-ad8f-a10edd1374b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:31.518799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c604035-3f27-4ea0-ad8f-a10edd1374b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:31.518799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c604035-3f27-4ea0-ad8f-a10edd1374b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:31.519845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c604035-3f27-4ea0-ad8f-a10edd1374b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:31.523224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c604035-3f27-4ea0-ad8f-a10edd1374b1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:31.551241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 920/1874] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:31.552021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438aa02-050b-4cb2-bf70-6eba3e48a9f2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:31.564653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438aa02-050b-4cb2-bf70-6eba3e48a9f2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:31.570200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc717ea2-881e-45c2-8844-6386e4796a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:31.571028 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc717ea2-881e-45c2-8844-6386e4796a78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:31.571501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc717ea2-881e-45c2-8844-6386e4796a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:31.573684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc717ea2-881e-45c2-8844-6386e4796a78 admin admin] PATH_INFO: `/v3/projects/d1daf8a697cd45baaebee128f04ea760/users/7932074e66454a938c1e6579a4d688ea/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:31.593503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c604035-3f27-4ea0-ad8f-a10edd1374b1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:31.596313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3c604035-3f27-4ea0-ad8f-a10edd1374b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:31.605345 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc717ea2-881e-45c2-8844-6386e4796a78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1daf8a697cd45baaebee128f04ea760, user_id=7932074e66454a938c1e6579a4d688ea, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:31.613815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c604035-3f27-4ea0-ad8f-a10edd1374b1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:31.619714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc717ea2-881e-45c2-8844-6386e4796a78 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:31.623955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:31.624495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:31.625780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:31.626143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] PATH_INFO: `/v3/projects/d42087027f5b4f43a96b61a01a960231` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:31.628222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d42087027f5b4f43a96b61a01a960231)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:31.630705 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:31.675136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 955/1875] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:31 2022] PUT /identity/v3/projects/d1daf8a697cd45baaebee128f04ea760/users/7932074e66454a938c1e6579a4d688ea/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:31.678495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc717ea2-881e-45c2-8844-6386e4796a78 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:31.770276 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42087027f5b4f43a96b61a01a960231', 'request_id': 'req-ba091d3c-239e-4f46-be78-41677b59ede9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:31.793859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42087027f5b4f43a96b61a01a960231', 'request_id': 'req-ba091d3c-239e-4f46-be78-41677b59ede9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:31.807701 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42087027f5b4f43a96b61a01a960231', 'request_id': 'req-ba091d3c-239e-4f46-be78-41677b59ede9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:31.834359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc717ea2-881e-45c2-8844-6386e4796a78 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:31.868420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] The token cache is being invalidate because project d42087027f5b4f43a96b61a01a960231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:31.868885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:31.877683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba091d3c-239e-4f46-be78-41677b59ede9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:31.885838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 921/1876] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:31 2022] DELETE /identity/v3/projects/d42087027f5b4f43a96b61a01a960231 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:31.902370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69eb83-0eba-4280-b5f2-662469aabcd5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:31.903089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69eb83-0eba-4280-b5f2-662469aabcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:31.903585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69eb83-0eba-4280-b5f2-662469aabcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:31.910769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:31.947077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8e69eb83-0eba-4280-b5f2-662469aabcd5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:31.949414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8e69eb83-0eba-4280-b5f2-662469aabcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:31.952513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1eff5-44c8-4f64-8c67-5c4262827364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:31.953464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1eff5-44c8-4f64-8c67-5c4262827364 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:31.953853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1eff5-44c8-4f64-8c67-5c4262827364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:31.954339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1eff5-44c8-4f64-8c67-5c4262827364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:31.955628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1eff5-44c8-4f64-8c67-5c4262827364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:31.969633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1eff5-44c8-4f64-8c67-5c4262827364 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:31.972340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 922/1877] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:31.987130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724a730-af8c-4706-a9b8-e2932c6410fd None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:31.987413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724a730-af8c-4706-a9b8-e2932c6410fd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:31.987696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724a730-af8c-4706-a9b8-e2932c6410fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:31.989943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 956/1878] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:31.995390 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1eff5-44c8-4f64-8c67-5c4262827364 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:31.998477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-12e1eff5-44c8-4f64-8c67-5c4262827364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:32.000883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:32.017330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0724a730-af8c-4706-a9b8-e2932c6410fd None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:32.019720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0724a730-af8c-4706-a9b8-e2932c6410fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:32.029741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1eff5-44c8-4f64-8c67-5c4262827364 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:32.037235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 923/1879] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:32.039239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724a730-af8c-4706-a9b8-e2932c6410fd None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:32.045420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32868ca3-0a43-497d-a650-014ca112e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:32.047447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32868ca3-0a43-497d-a650-014ca112e3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.048170 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32868ca3-0a43-497d-a650-014ca112e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.050559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32868ca3-0a43-497d-a650-014ca112e3a5 admin admin] PATH_INFO: `/v3/projects/d1daf8a697cd45baaebee128f04ea760/users/7932074e66454a938c1e6579a4d688ea/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.052583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724a730-af8c-4706-a9b8-e2932c6410fd None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:32.056826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32868ca3-0a43-497d-a650-014ca112e3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1daf8a697cd45baaebee128f04ea760, user_id=7932074e66454a938c1e6579a4d688ea, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:32.061518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89751d-ec66-4385-9008-2d3eac05329e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:32.062715 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89751d-ec66-4385-9008-2d3eac05329e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.063123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89751d-ec66-4385-9008-2d3eac05329e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.063696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89751d-ec66-4385-9008-2d3eac05329e admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.064176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32868ca3-0a43-497d-a650-014ca112e3a5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:32.065550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89751d-ec66-4385-9008-2d3eac05329e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:23:32.065850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89751d-ec66-4385-9008-2d3eac05329e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:32.068575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89751d-ec66-4385-9008-2d3eac05329e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:32.083698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 924/1880] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:23:32.096701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 957/1881] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:31 2022] PUT /identity/v3/projects/d1daf8a697cd45baaebee128f04ea760/users/7932074e66454a938c1e6579a4d688ea/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:32.096931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89751d-ec66-4385-9008-2d3eac05329e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:32.100194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072bdb-8ce1-4847-bbb1-2183755285e9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.100469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072bdb-8ce1-4847-bbb1-2183755285e9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.101065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072bdb-8ce1-4847-bbb1-2183755285e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.111844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89751d-ec66-4385-9008-2d3eac05329e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:32.135293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-92072bdb-8ce1-4847-bbb1-2183755285e9 None None] MFA Rules not processed for user `7932074e66454a938c1e6579a4d688ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:32.136010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-92072bdb-8ce1-4847-bbb1-2183755285e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:32.259348 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bba00b-b071-4036-9d84-e7200a6f02ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:32.260539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bba00b-b071-4036-9d84-e7200a6f02ad admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.261020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bba00b-b071-4036-9d84-e7200a6f02ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.261502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bba00b-b071-4036-9d84-e7200a6f02ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.263585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bba00b-b071-4036-9d84-e7200a6f02ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:32.267523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bba00b-b071-4036-9d84-e7200a6f02ad admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:32.331436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 925/1882] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:32 2022] POST /identity/v3/projects => generated 375 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:32.340020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bba00b-b071-4036-9d84-e7200a6f02ad admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:32.393661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 958/1883] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:32 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:32.415235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072bdb-8ce1-4847-bbb1-2183755285e9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:32.419325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-92072bdb-8ce1-4847-bbb1-2183755285e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:32.525790 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bba00b-b071-4036-9d84-e7200a6f02ad admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:32.562856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591e750-1231-4017-9e5b-452c33b76759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:32.564395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591e750-1231-4017-9e5b-452c33b76759 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.565087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591e750-1231-4017-9e5b-452c33b76759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.565514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072bdb-8ce1-4847-bbb1-2183755285e9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:32.567311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591e750-1231-4017-9e5b-452c33b76759 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.568707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591e750-1231-4017-9e5b-452c33b76759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:32.572903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591e750-1231-4017-9e5b-452c33b76759 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:32.628424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 926/1884] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:32 2022] POST /identity/v3/users => generated 288 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:23:32.630255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591e750-1231-4017-9e5b-452c33b76759 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:32.656961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591e750-1231-4017-9e5b-452c33b76759 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:32.671693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00f162-3583-4974-8094-e5b3559ec051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:32.673261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00f162-3583-4974-8094-e5b3559ec051 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.674021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00f162-3583-4974-8094-e5b3559ec051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.674699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00f162-3583-4974-8094-e5b3559ec051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.676256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00f162-3583-4974-8094-e5b3559ec051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:32.679637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00f162-3583-4974-8094-e5b3559ec051 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:32.705262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 927/1885] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:32.707183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30ffb5-da13-40ba-8eb8-f68ec3e997fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:32.707502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00f162-3583-4974-8094-e5b3559ec051 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:32.708286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30ffb5-da13-40ba-8eb8-f68ec3e997fd service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.709129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30ffb5-da13-40ba-8eb8-f68ec3e997fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.709609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30ffb5-da13-40ba-8eb8-f68ec3e997fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.733230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00f162-3583-4974-8094-e5b3559ec051 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:32.748763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:32.760640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa886e-0323-4d75-afe8-7ca6c845aede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:32.761323 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa886e-0323-4d75-afe8-7ca6c845aede admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.761614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa886e-0323-4d75-afe8-7ca6c845aede admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.762019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa886e-0323-4d75-afe8-7ca6c845aede admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.762750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa886e-0323-4d75-afe8-7ca6c845aede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:32.765291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa886e-0323-4d75-afe8-7ca6c845aede admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:32.797406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 928/1886] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:32.799182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa886e-0323-4d75-afe8-7ca6c845aede admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:32.812828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa886e-0323-4d75-afe8-7ca6c845aede admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:32.820798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf1eaf-cf4c-4fa8-8868-838ff82a7278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:32.822083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf1eaf-cf4c-4fa8-8868-838ff82a7278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.822371 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf1eaf-cf4c-4fa8-8868-838ff82a7278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.822680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf1eaf-cf4c-4fa8-8868-838ff82a7278 admin admin] PATH_INFO: `/v3/projects/ffdc7ce35fdd4310b3c1c1f19f01574a/users/9ba53391e2cf4db68ca01913570b8f64/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.843563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf1eaf-cf4c-4fa8-8868-838ff82a7278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdc7ce35fdd4310b3c1c1f19f01574a, user_id=9ba53391e2cf4db68ca01913570b8f64, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:32.847461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf1eaf-cf4c-4fa8-8868-838ff82a7278 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:32.859715 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30ffb5-da13-40ba-8eb8-f68ec3e997fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:32.863657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30ffb5-da13-40ba-8eb8-f68ec3e997fd service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:32.881123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 929/1887] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:32 2022] PUT /identity/v3/projects/ffdc7ce35fdd4310b3c1c1f19f01574a/users/9ba53391e2cf4db68ca01913570b8f64/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:32.883930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa231c-ad22-45fc-b5e7-808d74f559cb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:32.883930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa231c-ad22-45fc-b5e7-808d74f559cb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:32.884337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa231c-ad22-45fc-b5e7-808d74f559cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:32.885990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 959/1888] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:32 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:32.916704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:32.920893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-97fa231c-ad22-45fc-b5e7-808d74f559cb None None] MFA Rules not processed for user `9ba53391e2cf4db68ca01913570b8f64`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:32.922422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-97fa231c-ad22-45fc-b5e7-808d74f559cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:32.954129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30ffb5-da13-40ba-8eb8-f68ec3e997fd service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:32.956937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0f30ffb5-da13-40ba-8eb8-f68ec3e997fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:32.982906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30ffb5-da13-40ba-8eb8-f68ec3e997fd service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:33.070529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 930/1889] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:33.087737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa231c-ad22-45fc-b5e7-808d74f559cb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:33.089597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaf8b9-28bb-4840-8a83-028b9a82c2c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:33.089851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-97fa231c-ad22-45fc-b5e7-808d74f559cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:33.090325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaf8b9-28bb-4840-8a83-028b9a82c2c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:33.090609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaf8b9-28bb-4840-8a83-028b9a82c2c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:33.090887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaf8b9-28bb-4840-8a83-028b9a82c2c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:33.092908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8fcaf8b9-28bb-4840-8a83-028b9a82c2c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:33.186431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaf8b9-28bb-4840-8a83-028b9a82c2c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:33.189199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaf8b9-28bb-4840-8a83-028b9a82c2c5 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:33.189533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa231c-ad22-45fc-b5e7-808d74f559cb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:33.198618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacab081-5977-46a1-ac65-aea4139553a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41a357957bc47739b7bd6f21d159d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:33.199746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacab081-5977-46a1-ac65-aea4139553a5 service glance] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:33.200849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacab081-5977-46a1-ac65-aea4139553a5 service glance] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:33.201362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacab081-5977-46a1-ac65-aea4139553a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:33.208655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 960/1890] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:33.217916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacab081-5977-46a1-ac65-aea4139553a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:33.220591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacab081-5977-46a1-ac65-aea4139553a5 service glance] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:33.236844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 931/1891] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:23:38.245146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaf8b9-28bb-4840-8a83-028b9a82c2c5 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:38.363966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaf8b9-28bb-4840-8a83-028b9a82c2c5 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:38.369989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:38.370995 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:38.371417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:38.371699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] PATH_INFO: `/v3/users/74dcc98d05e34302a36bb7e4d78d8208` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:38.373760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74dcc98d05e34302a36bb7e4d78d8208)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:38.377300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:38.393289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:38.442850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74dcc98d05e34302a36bb7e4d78d8208'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:38.452496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74dcc98d05e34302a36bb7e4d78d8208'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:38.453610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74dcc98d05e34302a36bb7e4d78d8208'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:38.465940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74dcc98d05e34302a36bb7e4d78d8208'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:38.498877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74dcc98d05e34302a36bb7e4d78d8208'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:38.514609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 961/1892] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:38 2022] DELETE /identity/v3/users/74dcc98d05e34302a36bb7e4d78d8208 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:38.519280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a1e62-ca80-445c-8ffd-4d8c8c4a7dc1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:38.520363 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a1e62-ca80-445c-8ffd-4d8c8c4a7dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:38.520870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a1e62-ca80-445c-8ffd-4d8c8c4a7dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:38.544837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bc1a1e62-ca80-445c-8ffd-4d8c8c4a7dc1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:38.546858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bc1a1e62-ca80-445c-8ffd-4d8c8c4a7dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:38.644185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 932/1893] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:38.661398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:38.663553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:38.794698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d17d00-d1d6-4279-9d01-7f428fd35611 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:38.808613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec3e59-f671-439a-8a60-c4fe01b862a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:38.810365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec3e59-f671-439a-8a60-c4fe01b862a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:38.811030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec3e59-f671-439a-8a60-c4fe01b862a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:38.817328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec3e59-f671-439a-8a60-c4fe01b862a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:38.837753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec3e59-f671-439a-8a60-c4fe01b862a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:38.840778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec3e59-f671-439a-8a60-c4fe01b862a2 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:38.869869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 962/1894] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:38.891345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a1e62-ca80-445c-8ffd-4d8c8c4a7dc1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:38.892920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bc1a1e62-ca80-445c-8ffd-4d8c8c4a7dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:38.908018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a1e62-ca80-445c-8ffd-4d8c8c4a7dc1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:38.915780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:38.916849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:38.917213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:38.917546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] PATH_INFO: `/v3/users/1c74aa2746c845eea736dacf06a1a537` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:38.921803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c74aa2746c845eea736dacf06a1a537)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:38.923458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:39.029837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:39.046548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c74aa2746c845eea736dacf06a1a537'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.057816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c74aa2746c845eea736dacf06a1a537'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.059349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c74aa2746c845eea736dacf06a1a537'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.088669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c74aa2746c845eea736dacf06a1a537'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.090029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c74aa2746c845eea736dacf06a1a537'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.121789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 933/1895] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:38 2022] DELETE /identity/v3/users/1c74aa2746c845eea736dacf06a1a537 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:39.127828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04094640-67a9-4b94-8d0e-ce4e29f76d1d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:39.129053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04094640-67a9-4b94-8d0e-ce4e29f76d1d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:39.130721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04094640-67a9-4b94-8d0e-ce4e29f76d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:39.169009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-04094640-67a9-4b94-8d0e-ce4e29f76d1d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:39.171698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-04094640-67a9-4b94-8d0e-ce4e29f76d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:39.299454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 963/1896] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:39.306247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:39.308496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:39.334655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04094640-67a9-4b94-8d0e-ce4e29f76d1d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:39.335805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d724a-cc9d-4ed3-bf24-a01a90709b5d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:39.338589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-04094640-67a9-4b94-8d0e-ce4e29f76d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:39.351761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:39.353827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:39.354493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:39.355482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] PATH_INFO: `/v3/projects/319fc624b55e401085e554c42779ff26` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:39.358703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=319fc624b55e401085e554c42779ff26)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:39.363873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:39.439807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:39.485492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04094640-67a9-4b94-8d0e-ce4e29f76d1d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:39.504801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '319fc624b55e401085e554c42779ff26', 'request_id': 'req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.521237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '319fc624b55e401085e554c42779ff26', 'request_id': 'req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.545815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '319fc624b55e401085e554c42779ff26', 'request_id': 'req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.579564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] The token cache is being invalidate because project 319fc624b55e401085e554c42779ff26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:39.580095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.582045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.588761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 934/1897] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:39 2022] DELETE /identity/v3/projects/319fc624b55e401085e554c42779ff26 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:39.593430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:39.595426 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:39.636705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040683-7b48-4341-8cdb-f4b3b09a3d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:39.637624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040683-7b48-4341-8cdb-f4b3b09a3d79 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:39.638023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040683-7b48-4341-8cdb-f4b3b09a3d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:39.638376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040683-7b48-4341-8cdb-f4b3b09a3d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:39.640494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d4040683-7b48-4341-8cdb-f4b3b09a3d79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:39.714304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce075e33-fdeb-4d12-81c8-5e410f1d93c9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:39.720699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:39.721761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:39.722127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:39.722478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] PATH_INFO: `/v3/users/7932074e66454a938c1e6579a4d688ea` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:39.726844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7932074e66454a938c1e6579a4d688ea)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:39.729019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:39.801076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4040683-7b48-4341-8cdb-f4b3b09a3d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:39.803673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4040683-7b48-4341-8cdb-f4b3b09a3d79 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:39.816556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:39.822771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7932074e66454a938c1e6579a4d688ea'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.831682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 964/1898] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:39.846770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7932074e66454a938c1e6579a4d688ea'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.848845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7932074e66454a938c1e6579a4d688ea'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.908588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7932074e66454a938c1e6579a4d688ea'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.909411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7932074e66454a938c1e6579a4d688ea'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:39.941458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 935/1899] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:39 2022] DELETE /identity/v3/users/7932074e66454a938c1e6579a4d688ea => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:40.365474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040683-7b48-4341-8cdb-f4b3b09a3d79 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:40.396395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040683-7b48-4341-8cdb-f4b3b09a3d79 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:40.562336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:40.563252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:40.563539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:40.563874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] PATH_INFO: `/v3/projects/d1daf8a697cd45baaebee128f04ea760` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:40.567139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1daf8a697cd45baaebee128f04ea760)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:40.575104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:40.672294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:40.676843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:40.709904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:40.723550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1daf8a697cd45baaebee128f04ea760', 'request_id': 'req-5a6609b6-aeac-4add-9f52-f5825fad290d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:40.735428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929f599-ce2f-44f6-836c-f69c62cc149a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:40.735687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1daf8a697cd45baaebee128f04ea760', 'request_id': 'req-5a6609b6-aeac-4add-9f52-f5825fad290d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:40.750790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1daf8a697cd45baaebee128f04ea760', 'request_id': 'req-5a6609b6-aeac-4add-9f52-f5825fad290d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:40.767412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] The token cache is being invalidate because project d1daf8a697cd45baaebee128f04ea760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:40.767801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:40.768802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5a6609b6-aeac-4add-9f52-f5825fad290d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:40.771159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 965/1900] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:40 2022] DELETE /identity/v3/projects/d1daf8a697cd45baaebee128f04ea760 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:40.777801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f9d54-ce4b-46f9-82ee-563f594338d9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:40.778180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f9d54-ce4b-46f9-82ee-563f594338d9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:40.778767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f9d54-ce4b-46f9-82ee-563f594338d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:40.807409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c90f9d54-ce4b-46f9-82ee-563f594338d9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:40.809098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c90f9d54-ce4b-46f9-82ee-563f594338d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:40.881075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:40.884227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:40.884830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:40.887716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] PATH_INFO: `/v3/projects/9e585669e8ba49e684ca2c0913b8280f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:40.891949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e585669e8ba49e684ca2c0913b8280f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:40.896960 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:40.927152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 966/1901] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:40.929491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27b5de-0837-4bcf-ac05-885713bb4210 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:40.930051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27b5de-0837-4bcf-ac05-885713bb4210 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:40.930526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27b5de-0837-4bcf-ac05-885713bb4210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:40.966669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0c27b5de-0837-4bcf-ac05-885713bb4210 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:40.967900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0c27b5de-0837-4bcf-ac05-885713bb4210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:40.990419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 967/1902] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:40.997412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27b5de-0837-4bcf-ac05-885713bb4210 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:41.041548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27b5de-0837-4bcf-ac05-885713bb4210 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:41.067283 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e585669e8ba49e684ca2c0913b8280f', 'request_id': 'req-3e9acbc9-aac4-4215-8112-c09f51eaf44b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:41.081525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e585669e8ba49e684ca2c0913b8280f', 'request_id': 'req-3e9acbc9-aac4-4215-8112-c09f51eaf44b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:41.126836 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e585669e8ba49e684ca2c0913b8280f', 'request_id': 'req-3e9acbc9-aac4-4215-8112-c09f51eaf44b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:41.176903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] The token cache is being invalidate because project 9e585669e8ba49e684ca2c0913b8280f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:41.177849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:41.178650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3e9acbc9-aac4-4215-8112-c09f51eaf44b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:41.181281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 936/1903] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:40 2022] DELETE /identity/v3/projects/9e585669e8ba49e684ca2c0913b8280f => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:41.187875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af93f78-eee6-4e37-be96-675b0c1f8c3f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.187875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af93f78-eee6-4e37-be96-675b0c1f8c3f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.187875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af93f78-eee6-4e37-be96-675b0c1f8c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.202895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:41.202895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.203323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.203442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.204272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:23:41.204711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:41.206913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:41.232445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8af93f78-eee6-4e37-be96-675b0c1f8c3f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:41.236343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8af93f78-eee6-4e37-be96-675b0c1f8c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:41.250238 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 968/1904] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:23:41.255347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:41.257604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:41.262843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 937/1905] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:41.272611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81d3b9-ae32-46f9-b403-5bafca8d0c2f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.273088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81d3b9-ae32-46f9-b403-5bafca8d0c2f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.273536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81d3b9-ae32-46f9-b403-5bafca8d0c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.273731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd280f8-47e8-497e-9da4-bfea0edd631b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:41.295828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0df3-f3dc-4c8d-a8ff-5f0a088a0692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:41.296618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0df3-f3dc-4c8d-a8ff-5f0a088a0692 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.297056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0df3-f3dc-4c8d-a8ff-5f0a088a0692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.297494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0df3-f3dc-4c8d-a8ff-5f0a088a0692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.299360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a0df3-f3dc-4c8d-a8ff-5f0a088a0692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:41.300281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1d81d3b9-ae32-46f9-b403-5bafca8d0c2f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:41.302274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1d81d3b9-ae32-46f9-b403-5bafca8d0c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:41.303330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a0df3-f3dc-4c8d-a8ff-5f0a088a0692 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:41.314027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 938/1906] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:41.327223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81d3b9-ae32-46f9-b403-5bafca8d0c2f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:41.333080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81d3b9-ae32-46f9-b403-5bafca8d0c2f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:41.342962 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e2ea6-4b12-4a46-9968-88076421357d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:41.344143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2ea6-4b12-4a46-9968-88076421357d admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.344431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2ea6-4b12-4a46-9968-88076421357d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.344759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2ea6-4b12-4a46-9968-88076421357d admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.345513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e2ea6-4b12-4a46-9968-88076421357d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:23:41.346093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e2ea6-4b12-4a46-9968-88076421357d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:41.347708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e2ea6-4b12-4a46-9968-88076421357d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:41.352582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 969/1907] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:41 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:23:41.356659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0df3-f3dc-4c8d-a8ff-5f0a088a0692 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:41.379138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 939/1908] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:23:41.391732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e2ea6-4b12-4a46-9968-88076421357d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:41.425384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:41.444609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e2ea6-4b12-4a46-9968-88076421357d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:41.459236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0df3-f3dc-4c8d-a8ff-5f0a088a0692 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:41.494845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56de76-8556-43ee-878b-e26bfd4a04d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:41.495631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56de76-8556-43ee-878b-e26bfd4a04d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.495979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56de76-8556-43ee-878b-e26bfd4a04d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.496302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56de76-8556-43ee-878b-e26bfd4a04d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.497083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56de76-8556-43ee-878b-e26bfd4a04d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:41.499040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56de76-8556-43ee-878b-e26bfd4a04d2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:41.530575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:41.561490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 970/1909] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:41 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:41.563798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56de76-8556-43ee-878b-e26bfd4a04d2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:41.597039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56de76-8556-43ee-878b-e26bfd4a04d2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:41.615928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74f943-b656-409c-96c5-f29b0100f12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:41.618264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74f943-b656-409c-96c5-f29b0100f12f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.618792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74f943-b656-409c-96c5-f29b0100f12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.619292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74f943-b656-409c-96c5-f29b0100f12f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.620284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74f943-b656-409c-96c5-f29b0100f12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:41.622210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74f943-b656-409c-96c5-f29b0100f12f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:41.643478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 971/1910] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:41.644707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74f943-b656-409c-96c5-f29b0100f12f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:41.673068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74f943-b656-409c-96c5-f29b0100f12f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:41.685300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4606bb-b283-4149-ac00-051883ff7367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:41.686028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4606bb-b283-4149-ac00-051883ff7367 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.686268 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c58e6a-fb0f-488e-bc2c-544b3863b681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:41.686425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4606bb-b283-4149-ac00-051883ff7367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.686580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4606bb-b283-4149-ac00-051883ff7367 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.687302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4606bb-b283-4149-ac00-051883ff7367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:41.688092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c58e6a-fb0f-488e-bc2c-544b3863b681 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.688733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c58e6a-fb0f-488e-bc2c-544b3863b681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.689342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4606bb-b283-4149-ac00-051883ff7367 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:41.689649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c58e6a-fb0f-488e-bc2c-544b3863b681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.691472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c58e6a-fb0f-488e-bc2c-544b3863b681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:41.699759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c58e6a-fb0f-488e-bc2c-544b3863b681 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:41.730362 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 972/1911] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:41.734427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c58e6a-fb0f-488e-bc2c-544b3863b681 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:41.745257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 940/1912] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:41 2022] POST /identity/v3/projects => generated 363 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:41.751061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4606bb-b283-4149-ac00-051883ff7367 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:41.778904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c58e6a-fb0f-488e-bc2c-544b3863b681 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:41.922193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4606bb-b283-4149-ac00-051883ff7367 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:41.939373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cbb2c-a9ae-4866-8a6e-f09e2ee63721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:41.940696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cbb2c-a9ae-4866-8a6e-f09e2ee63721 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.941497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cbb2c-a9ae-4866-8a6e-f09e2ee63721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.942271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cbb2c-a9ae-4866-8a6e-f09e2ee63721 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.944130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cbb2c-a9ae-4866-8a6e-f09e2ee63721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:41.947966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a729f-e920-4184-95a2-8e853fd44e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:41.949033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a729f-e920-4184-95a2-8e853fd44e70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:41.949197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cbb2c-a9ae-4866-8a6e-f09e2ee63721 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:41.949342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a729f-e920-4184-95a2-8e853fd44e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:41.949588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a729f-e920-4184-95a2-8e853fd44e70 admin admin] PATH_INFO: `/v3/projects/bf9c854caae9424294769bf47bc4ddc5/users/7d8a90f6ed8b4368ac86d0d5b1f54fed/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:41.975461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a729f-e920-4184-95a2-8e853fd44e70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9c854caae9424294769bf47bc4ddc5, user_id=7d8a90f6ed8b4368ac86d0d5b1f54fed, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:41.980829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a729f-e920-4184-95a2-8e853fd44e70 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:42.021275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 973/1914] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:41 2022] PUT /identity/v3/projects/bf9c854caae9424294769bf47bc4ddc5/users/7d8a90f6ed8b4368ac86d0d5b1f54fed/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:42.021275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a729f-e920-4184-95a2-8e853fd44e70 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:42.022305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 941/1914] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:41 2022] POST /identity/v3/users => generated 282 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:42.029216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78a03b-77dc-4303-ac6a-b395b15e6adc None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.029675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78a03b-77dc-4303-ac6a-b395b15e6adc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.030099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78a03b-77dc-4303-ac6a-b395b15e6adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.111348 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-db78a03b-77dc-4303-ac6a-b395b15e6adc None None] MFA Rules not processed for user `7d8a90f6ed8b4368ac86d0d5b1f54fed`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:42.115987 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-db78a03b-77dc-4303-ac6a-b395b15e6adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:42.136606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a729f-e920-4184-95a2-8e853fd44e70 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:42.142788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3d0d4-43df-4227-8c6e-b71e03fc7995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:42.144306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3d0d4-43df-4227-8c6e-b71e03fc7995 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.144306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3d0d4-43df-4227-8c6e-b71e03fc7995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.144306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3d0d4-43df-4227-8c6e-b71e03fc7995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.145002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3d0d4-43df-4227-8c6e-b71e03fc7995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:42.158298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3d0d4-43df-4227-8c6e-b71e03fc7995 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:42.184290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 974/1915] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:42.206347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3d0d4-43df-4227-8c6e-b71e03fc7995 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:42.226901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3d0d4-43df-4227-8c6e-b71e03fc7995 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:42.243478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df9db7-f476-416b-85a4-da1b096888e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:42.244704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df9db7-f476-416b-85a4-da1b096888e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.250854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df9db7-f476-416b-85a4-da1b096888e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.250854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df9db7-f476-416b-85a4-da1b096888e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.252083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df9db7-f476-416b-85a4-da1b096888e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:42.259119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df9db7-f476-416b-85a4-da1b096888e4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:42.285601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 975/1916] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:42.293632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df9db7-f476-416b-85a4-da1b096888e4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:42.299150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 942/1917] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:42.303382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78a03b-77dc-4303-ac6a-b395b15e6adc None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:42.310119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df9db7-f476-416b-85a4-da1b096888e4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:42.318444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa36613-4c55-4842-acb7-3680c4f0b101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:42.319732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa36613-4c55-4842-acb7-3680c4f0b101 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.320218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa36613-4c55-4842-acb7-3680c4f0b101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.320677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa36613-4c55-4842-acb7-3680c4f0b101 admin admin] PATH_INFO: `/v3/projects/47fc06fdf6f447b3abd99c4d9b7a8183/users/1509082e02c74692855549955b3a8ffa/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.339386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78a03b-77dc-4303-ac6a-b395b15e6adc None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:42.343515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:42.348067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f10bce-1d26-4eb8-b89c-51c3bccecdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:42.350095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f10bce-1d26-4eb8-b89c-51c3bccecdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.350578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f10bce-1d26-4eb8-b89c-51c3bccecdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.351730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f10bce-1d26-4eb8-b89c-51c3bccecdb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.353375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f10bce-1d26-4eb8-b89c-51c3bccecdb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:42.357483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f10bce-1d26-4eb8-b89c-51c3bccecdb4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:42.372027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa36613-4c55-4842-acb7-3680c4f0b101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47fc06fdf6f447b3abd99c4d9b7a8183, user_id=1509082e02c74692855549955b3a8ffa, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:42.378054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa36613-4c55-4842-acb7-3680c4f0b101 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:42.427584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 976/1918] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:42 2022] PUT /identity/v3/projects/47fc06fdf6f447b3abd99c4d9b7a8183/users/1509082e02c74692855549955b3a8ffa/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:42.434068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cf51a-f4bf-4100-869f-217dd162ec80 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.436193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cf51a-f4bf-4100-869f-217dd162ec80 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.437087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cf51a-f4bf-4100-869f-217dd162ec80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.456455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 943/1919] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:42 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:23:42.461004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f10bce-1d26-4eb8-b89c-51c3bccecdb4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:42.480903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:42.519937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bb9cf51a-f4bf-4100-869f-217dd162ec80 None None] MFA Rules not processed for user `1509082e02c74692855549955b3a8ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:42.523515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bb9cf51a-f4bf-4100-869f-217dd162ec80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:42.634204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f10bce-1d26-4eb8-b89c-51c3bccecdb4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:42.641443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d6a60-184d-4909-956c-0e2fed536caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:42.643837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d6a60-184d-4909-956c-0e2fed536caa admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.644124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d6a60-184d-4909-956c-0e2fed536caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.644574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d6a60-184d-4909-956c-0e2fed536caa admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.645434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d6a60-184d-4909-956c-0e2fed536caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:42.649068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d6a60-184d-4909-956c-0e2fed536caa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:42.692125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 977/1920] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:42 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:42.701610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9cf51a-f4bf-4100-869f-217dd162ec80 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:42.703867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bb9cf51a-f4bf-4100-869f-217dd162ec80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:42.733975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 944/1921] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:42 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:42.739021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d6a60-184d-4909-956c-0e2fed536caa admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:42.763606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d6a60-184d-4909-956c-0e2fed536caa admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:42.779191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58af44-eeb0-48a8-bc29-c1b0871c06ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:42.781514 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58af44-eeb0-48a8-bc29-c1b0871c06ce admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.782065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58af44-eeb0-48a8-bc29-c1b0871c06ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.782535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58af44-eeb0-48a8-bc29-c1b0871c06ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.784217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58af44-eeb0-48a8-bc29-c1b0871c06ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:42.799584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58af44-eeb0-48a8-bc29-c1b0871c06ce admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:42.819656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9cf51a-f4bf-4100-869f-217dd162ec80 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:42.832119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 945/1922] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:42.834699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b585fcf-fe2e-4fbd-b077-b9bb540391cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:42.836757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58af44-eeb0-48a8-bc29-c1b0871c06ce admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:42.837547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b585fcf-fe2e-4fbd-b077-b9bb540391cc service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.837873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b585fcf-fe2e-4fbd-b077-b9bb540391cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.838154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b585fcf-fe2e-4fbd-b077-b9bb540391cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.851607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b585fcf-fe2e-4fbd-b077-b9bb540391cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:42.854929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b585fcf-fe2e-4fbd-b077-b9bb540391cc service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:42.872534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58af44-eeb0-48a8-bc29-c1b0871c06ce admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:42.879915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 978/1923] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:42 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:23:42.885263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a73e99-b1b4-4ef3-841d-4806e43cdc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:42.886935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a73e99-b1b4-4ef3-841d-4806e43cdc92 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.887287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a73e99-b1b4-4ef3-841d-4806e43cdc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.887862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a73e99-b1b4-4ef3-841d-4806e43cdc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:42.889358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a73e99-b1b4-4ef3-841d-4806e43cdc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:42.892810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a73e99-b1b4-4ef3-841d-4806e43cdc92 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:42.926458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 946/1924] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:42.932447 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b585fcf-fe2e-4fbd-b077-b9bb540391cc service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:42.962236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b585fcf-fe2e-4fbd-b077-b9bb540391cc service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:42.977227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac19483-ebfb-4de2-b0ca-ff917c2ab439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:42.984875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac19483-ebfb-4de2-b0ca-ff917c2ab439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:42.985355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac19483-ebfb-4de2-b0ca-ff917c2ab439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:42.986165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac19483-ebfb-4de2-b0ca-ff917c2ab439 admin admin] PATH_INFO: `/v3/projects/a0b96282cb6546db970763f1e6ab9785/users/d2b9062912df4c9ea95ddfd830e3a716/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:43.030034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac19483-ebfb-4de2-b0ca-ff917c2ab439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b96282cb6546db970763f1e6ab9785, user_id=d2b9062912df4c9ea95ddfd830e3a716, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:43.037615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac19483-ebfb-4de2-b0ca-ff917c2ab439 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:43.076415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 979/1925] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:42 2022] PUT /identity/v3/projects/a0b96282cb6546db970763f1e6ab9785/users/d2b9062912df4c9ea95ddfd830e3a716/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:43.089528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a2cd7-8ab2-49e2-963a-50ded96705c7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:43.090591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a2cd7-8ab2-49e2-963a-50ded96705c7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:43.091577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a2cd7-8ab2-49e2-963a-50ded96705c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:43.131270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-223a2cd7-8ab2-49e2-963a-50ded96705c7 None None] MFA Rules not processed for user `d2b9062912df4c9ea95ddfd830e3a716`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:43.133138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-223a2cd7-8ab2-49e2-963a-50ded96705c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:43.272286 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 947/1926] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:43.279621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac19483-ebfb-4de2-b0ca-ff917c2ab439 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:43.423762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:43.437364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac19483-ebfb-4de2-b0ca-ff917c2ab439 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:43.451858 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44941441-4138-4c73-aa88-90359339fcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:43.452773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44941441-4138-4c73-aa88-90359339fcfc admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:43.453071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44941441-4138-4c73-aa88-90359339fcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:43.453361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44941441-4138-4c73-aa88-90359339fcfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:43.454258 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44941441-4138-4c73-aa88-90359339fcfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:43.456272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44941441-4138-4c73-aa88-90359339fcfc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:43.533570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 980/1927] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:43 2022] POST /identity/v3/projects => generated 385 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:23:43.540175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a2cd7-8ab2-49e2-963a-50ded96705c7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:43.553321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a2cd7-8ab2-49e2-963a-50ded96705c7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:43.636754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a8cd9-9c10-405b-bd30-a0cf11700272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:43.637751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a8cd9-9c10-405b-bd30-a0cf11700272 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:43.637751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a8cd9-9c10-405b-bd30-a0cf11700272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:43.638086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a8cd9-9c10-405b-bd30-a0cf11700272 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:43.638710 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a8cd9-9c10-405b-bd30-a0cf11700272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:43.641181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a8cd9-9c10-405b-bd30-a0cf11700272 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:43.654903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:43.677565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 948/1928] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:43 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:43.683847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44941441-4138-4c73-aa88-90359339fcfc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:43.703650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44941441-4138-4c73-aa88-90359339fcfc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:43.710546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee2729-6b4c-405f-9850-f3bc64f74b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:43.711848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee2729-6b4c-405f-9850-f3bc64f74b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:43.712326 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee2729-6b4c-405f-9850-f3bc64f74b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:43.712729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee2729-6b4c-405f-9850-f3bc64f74b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:43.713603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee2729-6b4c-405f-9850-f3bc64f74b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:43.715552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee2729-6b4c-405f-9850-f3bc64f74b18 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:43.732222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 981/1929] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:43.738411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a8cd9-9c10-405b-bd30-a0cf11700272 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:43.751774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a8cd9-9c10-405b-bd30-a0cf11700272 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:43.765767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5536c-f7dc-40c4-9a86-37e8e303500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:43.767548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5536c-f7dc-40c4-9a86-37e8e303500d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:43.767966 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5536c-f7dc-40c4-9a86-37e8e303500d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:43.768390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5536c-f7dc-40c4-9a86-37e8e303500d admin admin] PATH_INFO: `/v3/projects/916c5b3ffdbd4c50b303597e072b3c29/users/7e5ef413926a4998855b193d0e832d6b/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:43.790468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5536c-f7dc-40c4-9a86-37e8e303500d admin admin] RBAC: Authorizing `identity:create_grant(project_id=916c5b3ffdbd4c50b303597e072b3c29, user_id=7e5ef413926a4998855b193d0e832d6b, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:43.793626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5536c-f7dc-40c4-9a86-37e8e303500d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:43.810746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 949/1930] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:43 2022] PUT /identity/v3/projects/916c5b3ffdbd4c50b303597e072b3c29/users/7e5ef413926a4998855b193d0e832d6b/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:43.815162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee2729-6b4c-405f-9850-f3bc64f74b18 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:43.828119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee2729-6b4c-405f-9850-f3bc64f74b18 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:43.899080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aff886-af69-4488-be85-32d4a198053e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:43.900017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aff886-af69-4488-be85-32d4a198053e admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:43.900446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aff886-af69-4488-be85-32d4a198053e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:43.900777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aff886-af69-4488-be85-32d4a198053e admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:43.901612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aff886-af69-4488-be85-32d4a198053e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:43.903590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aff886-af69-4488-be85-32d4a198053e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:43.917561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 982/1931] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:43.922847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5536c-f7dc-40c4-9a86-37e8e303500d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:43.936037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5536c-f7dc-40c4-9a86-37e8e303500d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:43.941807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0e9df-0389-47fe-b4dd-0dd45e2d0207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:43.942737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0e9df-0389-47fe-b4dd-0dd45e2d0207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:43.943017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0e9df-0389-47fe-b4dd-0dd45e2d0207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:43.943291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0e9df-0389-47fe-b4dd-0dd45e2d0207 admin admin] PATH_INFO: `/v3/projects/916c5b3ffdbd4c50b303597e072b3c29/users/7e5ef413926a4998855b193d0e832d6b/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:43.946106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0e9df-0389-47fe-b4dd-0dd45e2d0207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=916c5b3ffdbd4c50b303597e072b3c29, user_id=7e5ef413926a4998855b193d0e832d6b, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:43.948862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0e9df-0389-47fe-b4dd-0dd45e2d0207 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:43.962666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 950/1932] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:43 2022] PUT /identity/v3/projects/916c5b3ffdbd4c50b303597e072b3c29/users/7e5ef413926a4998855b193d0e832d6b/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:43.974233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2a1e-8500-45f9-a086-b41798513816 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:43.974540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2a1e-8500-45f9-a086-b41798513816 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:43.974908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2a1e-8500-45f9-a086-b41798513816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:44.009266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-680a2a1e-8500-45f9-a086-b41798513816 None None] MFA Rules not processed for user `7e5ef413926a4998855b193d0e832d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:44.011003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-680a2a1e-8500-45f9-a086-b41798513816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:44.142953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 983/1933] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:44.159413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0e9df-0389-47fe-b4dd-0dd45e2d0207 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:44.294545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0e9df-0389-47fe-b4dd-0dd45e2d0207 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:44.299367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a94713-19d2-4c68-8991-d7e1d5dcb10c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:44.300428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a94713-19d2-4c68-8991-d7e1d5dcb10c service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:44.301026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a94713-19d2-4c68-8991-d7e1d5dcb10c service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:44.301950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a94713-19d2-4c68-8991-d7e1d5dcb10c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:44.317651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a94713-19d2-4c68-8991-d7e1d5dcb10c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:44.321162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a94713-19d2-4c68-8991-d7e1d5dcb10c service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:44.337992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 951/1934] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:45.212380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a2a1e-8500-45f9-a086-b41798513816 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:45.237850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a2a1e-8500-45f9-a086-b41798513816 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:45.244441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae621c7e-ebd4-4fde-ac70-da4a53c4f36a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:45.245320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae621c7e-ebd4-4fde-ac70-da4a53c4f36a service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:45.245879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae621c7e-ebd4-4fde-ac70-da4a53c4f36a service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:45.246290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae621c7e-ebd4-4fde-ac70-da4a53c4f36a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:45.322850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:45.336383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae621c7e-ebd4-4fde-ac70-da4a53c4f36a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:45.338178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae621c7e-ebd4-4fde-ac70-da4a53c4f36a service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:45.361907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 984/1935] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:47.524511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010942e3-daf9-410a-a1a0-85b8b992ee7c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:47.525892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010942e3-daf9-410a-a1a0-85b8b992ee7c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:47.526433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010942e3-daf9-410a-a1a0-85b8b992ee7c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:47.528062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 952/1936] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:23:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:23:47.544094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3370-a94c-4ebf-801a-a184e6cc9b07 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:47.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3370-a94c-4ebf-801a-a184e6cc9b07 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:47.545668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3370-a94c-4ebf-801a-a184e6cc9b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:47.569062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d13b3370-a94c-4ebf-801a-a184e6cc9b07 None None] MFA Rules not processed for user `51b91456a0634e0fa22fb389f0721a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:47.572104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d13b3370-a94c-4ebf-801a-a184e6cc9b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:47.647088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010942e3-daf9-410a-a1a0-85b8b992ee7c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:47.649500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-010942e3-daf9-410a-a1a0-85b8b992ee7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:47.678048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 985/1937] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:23:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:47.690457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b3370-a94c-4ebf-801a-a184e6cc9b07 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:47.691879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d13b3370-a94c-4ebf-801a-a184e6cc9b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:47.707542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b3370-a94c-4ebf-801a-a184e6cc9b07 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:47.718099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6affda8-2444-421c-9331-dc0f98b71add service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:47.719209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6affda8-2444-421c-9331-dc0f98b71add service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:47.719733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6affda8-2444-421c-9331-dc0f98b71add service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:47.720253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6affda8-2444-421c-9331-dc0f98b71add service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:47.733397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6affda8-2444-421c-9331-dc0f98b71add service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:47.735074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6affda8-2444-421c-9331-dc0f98b71add service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:47.745779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010942e3-daf9-410a-a1a0-85b8b992ee7c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:47.749314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 986/1938] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:23:47.750623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:47.751621 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:47.751940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:47.752252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] PATH_INFO: `/v3/users/a5fee90ee6694ae4a821f60ca194ce84` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:47.753734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5fee90ee6694ae4a821f60ca194ce84)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:47.755927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:47.821288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5fee90ee6694ae4a821f60ca194ce84'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:47.840763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5fee90ee6694ae4a821f60ca194ce84'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:47.842350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5fee90ee6694ae4a821f60ca194ce84'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:47.850464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:47.864681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5fee90ee6694ae4a821f60ca194ce84'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:47.866073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-137accd1-7da3-4b30-9df6-5a6a3459f2bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5fee90ee6694ae4a821f60ca194ce84'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:47.897046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 953/1939] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:47 2022] DELETE /identity/v3/users/a5fee90ee6694ae4a821f60ca194ce84 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:48.161263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6affda8-2444-421c-9331-dc0f98b71add service placement] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:48.197433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6affda8-2444-421c-9331-dc0f98b71add service placement] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:48.299326 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:48.300948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:48.301455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:48.302009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] PATH_INFO: `/v3/projects/c2ab6d13d539433a853fc6805d819b85` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:48.306186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ab6d13d539433a853fc6805d819b85)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:48.308476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:48.383693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ab6d13d539433a853fc6805d819b85', 'request_id': 'req-b3c9a96e-c181-4d3e-9558-b910f4e99127'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:48.392756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ab6d13d539433a853fc6805d819b85', 'request_id': 'req-b3c9a96e-c181-4d3e-9558-b910f4e99127'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:48.405450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ab6d13d539433a853fc6805d819b85', 'request_id': 'req-b3c9a96e-c181-4d3e-9558-b910f4e99127'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:48.420775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] The token cache is being invalidate because project c2ab6d13d539433a853fc6805d819b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:48.421093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:48.421691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3c9a96e-c181-4d3e-9558-b910f4e99127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:48.426998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 987/1940] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:48 2022] DELETE /identity/v3/projects/c2ab6d13d539433a853fc6805d819b85 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:48.430286 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9f3d7-cbcb-4169-965c-a4e5958a68d0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:48.430769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9f3d7-cbcb-4169-965c-a4e5958a68d0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:48.431252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9f3d7-cbcb-4169-965c-a4e5958a68d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:48.462933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-dff9f3d7-cbcb-4169-965c-a4e5958a68d0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:48.465311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dff9f3d7-cbcb-4169-965c-a4e5958a68d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:48.572699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 954/1941] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:48.581752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe174e-7e36-4967-8141-199cb8c0f2a3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:48.582937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe174e-7e36-4967-8141-199cb8c0f2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:48.583227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe174e-7e36-4967-8141-199cb8c0f2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:48.595569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:48.614540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1ffe174e-7e36-4967-8141-199cb8c0f2a3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:48.629734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1ffe174e-7e36-4967-8141-199cb8c0f2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:48.640186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 988/1942] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:48.645573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9f3d7-cbcb-4169-965c-a4e5958a68d0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:48.664675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9f3d7-cbcb-4169-965c-a4e5958a68d0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:48.673506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0fbc9-b08d-4c25-ad85-a49efcedec91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:48.674709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0fbc9-b08d-4c25-ad85-a49efcedec91 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:48.675157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0fbc9-b08d-4c25-ad85-a49efcedec91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:48.675649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0fbc9-b08d-4c25-ad85-a49efcedec91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:48.676859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0fbc9-b08d-4c25-ad85-a49efcedec91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:23:48.677273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0fbc9-b08d-4c25-ad85-a49efcedec91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:48.679842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0fbc9-b08d-4c25-ad85-a49efcedec91 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:48.694407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 955/1943] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:23:48.699639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe174e-7e36-4967-8141-199cb8c0f2a3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:48.712818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe174e-7e36-4967-8141-199cb8c0f2a3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:48.718025 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330a948-4951-489d-ae64-c60506031000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:48.718724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330a948-4951-489d-ae64-c60506031000 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:48.719013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330a948-4951-489d-ae64-c60506031000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:48.719368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330a948-4951-489d-ae64-c60506031000 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:48.720152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330a948-4951-489d-ae64-c60506031000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:48.722352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330a948-4951-489d-ae64-c60506031000 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:48.755431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 989/1944] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:48 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:48.760765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0fbc9-b08d-4c25-ad85-a49efcedec91 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:48.778901 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0fbc9-b08d-4c25-ad85-a49efcedec91 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:48.856455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d217d-b246-4fab-bcd2-23da2f4d1b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:48.857165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d217d-b246-4fab-bcd2-23da2f4d1b79 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:48.857447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d217d-b246-4fab-bcd2-23da2f4d1b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:48.857763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d217d-b246-4fab-bcd2-23da2f4d1b79 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:48.858462 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d217d-b246-4fab-bcd2-23da2f4d1b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:48.860431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d217d-b246-4fab-bcd2-23da2f4d1b79 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:48.884059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 956/1945] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:48 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:48.889444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330a948-4951-489d-ae64-c60506031000 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:48.905984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330a948-4951-489d-ae64-c60506031000 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:48.916644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44649f-e02e-42f2-b60c-561df2c3a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:48.917807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44649f-e02e-42f2-b60c-561df2c3a293 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:48.918280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44649f-e02e-42f2-b60c-561df2c3a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:48.918743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44649f-e02e-42f2-b60c-561df2c3a293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:48.920017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44649f-e02e-42f2-b60c-561df2c3a293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:48.923135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44649f-e02e-42f2-b60c-561df2c3a293 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:48.944428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 990/1946] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:48.949786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d217d-b246-4fab-bcd2-23da2f4d1b79 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:48.963640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d217d-b246-4fab-bcd2-23da2f4d1b79 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:48.969965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e1d8f-2074-4ba7-8562-4c2bb3607382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:48.970939 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e1d8f-2074-4ba7-8562-4c2bb3607382 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:48.971316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e1d8f-2074-4ba7-8562-4c2bb3607382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:48.971708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e1d8f-2074-4ba7-8562-4c2bb3607382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:48.972553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e1d8f-2074-4ba7-8562-4c2bb3607382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:48.974335 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e1d8f-2074-4ba7-8562-4c2bb3607382 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:48.988667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 957/1947] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:23:48.994422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44649f-e02e-42f2-b60c-561df2c3a293 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:49.006371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44649f-e02e-42f2-b60c-561df2c3a293 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:49.011796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e656a-2514-474d-9dfe-a137cd7a5dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:49.012733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e656a-2514-474d-9dfe-a137cd7a5dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:49.013020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e656a-2514-474d-9dfe-a137cd7a5dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:49.013302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e656a-2514-474d-9dfe-a137cd7a5dc9 admin admin] PATH_INFO: `/v3/projects/5963ca6e05ba42c4993767cf1c269438/users/041bdd97691647e893e56ae2184c0271/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:49.031433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e656a-2514-474d-9dfe-a137cd7a5dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5963ca6e05ba42c4993767cf1c269438, user_id=041bdd97691647e893e56ae2184c0271, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:49.034548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e656a-2514-474d-9dfe-a137cd7a5dc9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:49.050372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 991/1948] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:48 2022] PUT /identity/v3/projects/5963ca6e05ba42c4993767cf1c269438/users/041bdd97691647e893e56ae2184c0271/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:49.062058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bf3af-8a8e-469e-a0c5-230746a131d3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:49.062471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bf3af-8a8e-469e-a0c5-230746a131d3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:49.062805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bf3af-8a8e-469e-a0c5-230746a131d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:49.097061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-536bf3af-8a8e-469e-a0c5-230746a131d3 None None] MFA Rules not processed for user `041bdd97691647e893e56ae2184c0271`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:49.098334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-536bf3af-8a8e-469e-a0c5-230746a131d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:49.144142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:49.191087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 958/1949] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:49.195847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e656a-2514-474d-9dfe-a137cd7a5dc9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:49.305271 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e656a-2514-474d-9dfe-a137cd7a5dc9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:49.314477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba57a48-4289-4ee9-ba04-a77b3e76ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:49.315569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba57a48-4289-4ee9-ba04-a77b3e76ec26 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:49.316044 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba57a48-4289-4ee9-ba04-a77b3e76ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:49.316504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba57a48-4289-4ee9-ba04-a77b3e76ec26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:49.317400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba57a48-4289-4ee9-ba04-a77b3e76ec26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:49.320497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba57a48-4289-4ee9-ba04-a77b3e76ec26 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:49.370892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 992/1950] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:49 2022] POST /identity/v3/projects => generated 369 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:23:49.376301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bf3af-8a8e-469e-a0c5-230746a131d3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:49.390562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bf3af-8a8e-469e-a0c5-230746a131d3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:49.533946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0adc62-df75-4c6d-bf57-3c9a28b2f51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:49.535320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0adc62-df75-4c6d-bf57-3c9a28b2f51c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:49.535776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0adc62-df75-4c6d-bf57-3c9a28b2f51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:49.536227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0adc62-df75-4c6d-bf57-3c9a28b2f51c admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:49.537425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0adc62-df75-4c6d-bf57-3c9a28b2f51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:49.542124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0adc62-df75-4c6d-bf57-3c9a28b2f51c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:49.577782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 959/1951] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:49 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:49.581754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba57a48-4289-4ee9-ba04-a77b3e76ec26 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:49.592893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba57a48-4289-4ee9-ba04-a77b3e76ec26 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:49.601226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b896256-5f37-4ddd-868d-96b8e6fc38e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:49.602560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b896256-5f37-4ddd-868d-96b8e6fc38e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:49.603115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b896256-5f37-4ddd-868d-96b8e6fc38e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:49.603695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b896256-5f37-4ddd-868d-96b8e6fc38e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:49.621151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b896256-5f37-4ddd-868d-96b8e6fc38e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:49.621921 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b896256-5f37-4ddd-868d-96b8e6fc38e4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:49.646317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 993/1952] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:23:49.654406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0adc62-df75-4c6d-bf57-3c9a28b2f51c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:49.677165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0adc62-df75-4c6d-bf57-3c9a28b2f51c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:49.686729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20342c-68dd-4b35-a7e5-b331c5097807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:49.688914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20342c-68dd-4b35-a7e5-b331c5097807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:49.689507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20342c-68dd-4b35-a7e5-b331c5097807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:49.690118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20342c-68dd-4b35-a7e5-b331c5097807 admin admin] PATH_INFO: `/v3/projects/ff9b740bf98a4f72af048e0105f603e2/users/ca5dd913f12740098c7148c4aadd77c5/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:49.725250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20342c-68dd-4b35-a7e5-b331c5097807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9b740bf98a4f72af048e0105f603e2, user_id=ca5dd913f12740098c7148c4aadd77c5, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:49.731906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20342c-68dd-4b35-a7e5-b331c5097807 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:49.755226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 960/1953] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:49 2022] PUT /identity/v3/projects/ff9b740bf98a4f72af048e0105f603e2/users/ca5dd913f12740098c7148c4aadd77c5/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:49.759966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b896256-5f37-4ddd-868d-96b8e6fc38e4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:49.780449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b896256-5f37-4ddd-868d-96b8e6fc38e4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:49.906004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264da31-c36e-425b-8dd0-346e2ee27866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:49.906700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264da31-c36e-425b-8dd0-346e2ee27866 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:49.906991 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264da31-c36e-425b-8dd0-346e2ee27866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:49.907318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264da31-c36e-425b-8dd0-346e2ee27866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:49.908033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264da31-c36e-425b-8dd0-346e2ee27866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:49.910214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264da31-c36e-425b-8dd0-346e2ee27866 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:49.924129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 994/1954] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:23:49.928261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20342c-68dd-4b35-a7e5-b331c5097807 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:49.940779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20342c-68dd-4b35-a7e5-b331c5097807 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:49.946543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed788506-5012-441b-8a0e-335dd3468289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:49.948334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788506-5012-441b-8a0e-335dd3468289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:49.948906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788506-5012-441b-8a0e-335dd3468289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:49.951880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed788506-5012-441b-8a0e-335dd3468289 admin admin] PATH_INFO: `/v3/projects/ff9b740bf98a4f72af048e0105f603e2/users/ca5dd913f12740098c7148c4aadd77c5/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:49.954745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed788506-5012-441b-8a0e-335dd3468289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9b740bf98a4f72af048e0105f603e2, user_id=ca5dd913f12740098c7148c4aadd77c5, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:49.957907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed788506-5012-441b-8a0e-335dd3468289 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:49.978508 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 961/1955] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:49 2022] PUT /identity/v3/projects/ff9b740bf98a4f72af048e0105f603e2/users/ca5dd913f12740098c7148c4aadd77c5/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:49.998173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e3be0-f08f-4e0b-9e63-2e76a86767c1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:49.998921 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e3be0-f08f-4e0b-9e63-2e76a86767c1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:49.999467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e3be0-f08f-4e0b-9e63-2e76a86767c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:50.011630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed788506-5012-441b-8a0e-335dd3468289 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:50.014794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ed788506-5012-441b-8a0e-335dd3468289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:50.029814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:50.049287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a47e3be0-f08f-4e0b-9e63-2e76a86767c1 None None] MFA Rules not processed for user `ca5dd913f12740098c7148c4aadd77c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:50.051329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a47e3be0-f08f-4e0b-9e63-2e76a86767c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:50.107773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed788506-5012-441b-8a0e-335dd3468289 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:50.111802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:50.112490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:50.112773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:50.113061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:50.116081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:50.159053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 995/1956] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:49 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:50.173128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e3be0-f08f-4e0b-9e63-2e76a86767c1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:50.175280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a47e3be0-f08f-4e0b-9e63-2e76a86767c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:50.190920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:50.195472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e3be0-f08f-4e0b-9e63-2e76a86767c1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:50.201920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2dfc2-14bd-4f08-aefc-669e6537690c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:50.202943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2dfc2-14bd-4f08-aefc-669e6537690c service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:50.203555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2dfc2-14bd-4f08-aefc-669e6537690c service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:50.204014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2dfc2-14bd-4f08-aefc-669e6537690c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:50.217350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2dfc2-14bd-4f08-aefc-669e6537690c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:50.217888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:50.219446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2dfc2-14bd-4f08-aefc-669e6537690c service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:50.220430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:50.233217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 996/1957] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:50 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:23:50.235155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 962/1958] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:23:50.324929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2dfc2-14bd-4f08-aefc-669e6537690c service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:50.335181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2dfc2-14bd-4f08-aefc-669e6537690c service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:50.341242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689f86a-278a-4c57-a51d-1e899358d061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:50.342546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689f86a-278a-4c57-a51d-1e899358d061 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:50.343142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689f86a-278a-4c57-a51d-1e899358d061 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:50.343955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689f86a-278a-4c57-a51d-1e899358d061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:50.426089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689f86a-278a-4c57-a51d-1e899358d061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:50.427750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689f86a-278a-4c57-a51d-1e899358d061 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:50.440073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 997/1959] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:52.911955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:52.914975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:52.996191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d31ff-2697-484f-9d98-1b5e9d3c527e service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:53.001208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:53.002163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:53.002440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:53.002709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] PATH_INFO: `/v3/users/7d8a90f6ed8b4368ac86d0d5b1f54fed` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:53.004003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d8a90f6ed8b4368ac86d0d5b1f54fed)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:53.006032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:53.067859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d8a90f6ed8b4368ac86d0d5b1f54fed'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.078465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d8a90f6ed8b4368ac86d0d5b1f54fed'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.079474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d8a90f6ed8b4368ac86d0d5b1f54fed'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.097162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d8a90f6ed8b4368ac86d0d5b1f54fed'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.098204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d8a90f6ed8b4368ac86d0d5b1f54fed'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.119490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 963/1960] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:52 2022] DELETE /identity/v3/users/7d8a90f6ed8b4368ac86d0d5b1f54fed => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:53.125425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62a77-0a14-45a3-afd2-1d1fa9c75bfc None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:53.126054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62a77-0a14-45a3-afd2-1d1fa9c75bfc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:53.126450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62a77-0a14-45a3-afd2-1d1fa9c75bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:53.148707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-eed62a77-0a14-45a3-afd2-1d1fa9c75bfc None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:53.150175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-eed62a77-0a14-45a3-afd2-1d1fa9c75bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:53.246692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 998/1961] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:53.263213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:53.266135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:53.388886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd34ec1-5d4a-43ed-9aaf-ff31bbe7e98b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:53.393060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb8023-5eeb-4671-bef4-329640744214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:53.394126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb8023-5eeb-4671-bef4-329640744214 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:53.394126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb8023-5eeb-4671-bef4-329640744214 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:53.394330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb8023-5eeb-4671-bef4-329640744214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:53.405298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb8023-5eeb-4671-bef4-329640744214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:53.406897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb8023-5eeb-4671-bef4-329640744214 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:53.418657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 964/1962] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:53.821097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62a77-0a14-45a3-afd2-1d1fa9c75bfc None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:53.823705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-eed62a77-0a14-45a3-afd2-1d1fa9c75bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:53.843143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62a77-0a14-45a3-afd2-1d1fa9c75bfc None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:53.850615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:53.851965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:53.852396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:53.853053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] PATH_INFO: `/v3/projects/bf9c854caae9424294769bf47bc4ddc5` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:53.855341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf9c854caae9424294769bf47bc4ddc5)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:53.858528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:53.899307 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:53.940642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9c854caae9424294769bf47bc4ddc5', 'request_id': 'req-2813191e-7f50-4b91-855d-43fc3ac286f5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.951843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9c854caae9424294769bf47bc4ddc5', 'request_id': 'req-2813191e-7f50-4b91-855d-43fc3ac286f5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.961106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9c854caae9424294769bf47bc4ddc5', 'request_id': 'req-2813191e-7f50-4b91-855d-43fc3ac286f5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.974136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] The token cache is being invalidate because project bf9c854caae9424294769bf47bc4ddc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:53.974520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.975778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:53.977568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 999/1963] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:53 2022] DELETE /identity/v3/projects/bf9c854caae9424294769bf47bc4ddc5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:53.982048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb8023-5eeb-4671-bef4-329640744214 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:53.987573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:53.997981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb8023-5eeb-4671-bef4-329640744214 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:54.066379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:54.067340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:54.067613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:54.067918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] PATH_INFO: `/v3/users/d2b9062912df4c9ea95ddfd830e3a716` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:54.069252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b9062912df4c9ea95ddfd830e3a716)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:54.071577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:54.128601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b9062912df4c9ea95ddfd830e3a716'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.142727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b9062912df4c9ea95ddfd830e3a716'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.143914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b9062912df4c9ea95ddfd830e3a716'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.168224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b9062912df4c9ea95ddfd830e3a716'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.169410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b9062912df4c9ea95ddfd830e3a716'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.188150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 965/1964] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:53 2022] DELETE /identity/v3/users/d2b9062912df4c9ea95ddfd830e3a716 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:54.663514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:54.666694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:54.681880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813191e-7f50-4b91-855d-43fc3ac286f5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:54.774506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:54.776102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:54.776620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:54.777132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] PATH_INFO: `/v3/projects/a0b96282cb6546db970763f1e6ab9785` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:54.779655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b96282cb6546db970763f1e6ab9785)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:54.783728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:54.862298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b96282cb6546db970763f1e6ab9785', 'request_id': 'req-f0acc1b2-aaa3-4127-9680-223894418f0e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.872336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b96282cb6546db970763f1e6ab9785', 'request_id': 'req-f0acc1b2-aaa3-4127-9680-223894418f0e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.879918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b96282cb6546db970763f1e6ab9785', 'request_id': 'req-f0acc1b2-aaa3-4127-9680-223894418f0e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.893215 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] The token cache is being invalidate because project a0b96282cb6546db970763f1e6ab9785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:54.893510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.894336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:54.895959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1000/1965] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:54 2022] DELETE /identity/v3/projects/a0b96282cb6546db970763f1e6ab9785 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:54.902545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:54.904167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:54.920321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45649b-efc9-48fe-a0f4-554530c8f8c4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:54.992292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:54.993346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:54.993627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:54.993961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] PATH_INFO: `/v3/users/7e5ef413926a4998855b193d0e832d6b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:54.995327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5ef413926a4998855b193d0e832d6b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:54.997290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:55.027590 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:55.069023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5ef413926a4998855b193d0e832d6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.081995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5ef413926a4998855b193d0e832d6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.083161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5ef413926a4998855b193d0e832d6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.108397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5ef413926a4998855b193d0e832d6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.109530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e8bba0f2-3541-43db-95db-88dd5d136c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5ef413926a4998855b193d0e832d6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.128629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 966/1966] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:23:54 2022] DELETE /identity/v3/users/7e5ef413926a4998855b193d0e832d6b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:55.472118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:55.473650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:55.490456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acc1b2-aaa3-4127-9680-223894418f0e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:55.536109 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:55.574465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:55.575104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:55.575396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:55.575909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] PATH_INFO: `/v3/projects/916c5b3ffdbd4c50b303597e072b3c29` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:55.577251 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=916c5b3ffdbd4c50b303597e072b3c29)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:55.579424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:55.649485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '916c5b3ffdbd4c50b303597e072b3c29', 'request_id': 'req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.660187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '916c5b3ffdbd4c50b303597e072b3c29', 'request_id': 'req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.673818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '916c5b3ffdbd4c50b303597e072b3c29', 'request_id': 'req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.683185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] The token cache is being invalidate because project 916c5b3ffdbd4c50b303597e072b3c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:23:55.683477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.684763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6edc9ad8-099f-44cc-88ef-c890bc2d2f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:23:55.685957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1001/1967] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:23:55 2022] DELETE /identity/v3/projects/916c5b3ffdbd4c50b303597e072b3c29 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:55.692920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130c17e-fb43-41d6-b8c0-bd79cae3d2fa None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:55.693791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130c17e-fb43-41d6-b8c0-bd79cae3d2fa None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:55.694586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130c17e-fb43-41d6-b8c0-bd79cae3d2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:55.729465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6130c17e-fb43-41d6-b8c0-bd79cae3d2fa None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:55.732130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6130c17e-fb43-41d6-b8c0-bd79cae3d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:55.842628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 967/1968] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:55.846229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a902114-bd8d-4f78-92e8-514673e246f0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:55.846759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a902114-bd8d-4f78-92e8-514673e246f0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:55.847278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a902114-bd8d-4f78-92e8-514673e246f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:55.883488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2a902114-bd8d-4f78-92e8-514673e246f0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:55.886612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2a902114-bd8d-4f78-92e8-514673e246f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:55.901401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1002/1969] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:23:55.906629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130c17e-fb43-41d6-b8c0-bd79cae3d2fa None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:55.926047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130c17e-fb43-41d6-b8c0-bd79cae3d2fa None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:55.932708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4eec8e-ade2-497c-95c1-a15042553344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:55.933670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4eec8e-ade2-497c-95c1-a15042553344 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:55.934131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4eec8e-ade2-497c-95c1-a15042553344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:55.934574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4eec8e-ade2-497c-95c1-a15042553344 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:55.935736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4eec8e-ade2-497c-95c1-a15042553344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:23:55.936244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4eec8e-ade2-497c-95c1-a15042553344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:55.939454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4eec8e-ade2-497c-95c1-a15042553344 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:55.955797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 968/1970] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:23:55.961214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a902114-bd8d-4f78-92e8-514673e246f0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:55.971552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a902114-bd8d-4f78-92e8-514673e246f0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:55.977377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bebeb-aaf0-4279-90f9-396ea4dab547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:55.978128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bebeb-aaf0-4279-90f9-396ea4dab547 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:55.978418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bebeb-aaf0-4279-90f9-396ea4dab547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:55.978706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bebeb-aaf0-4279-90f9-396ea4dab547 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:55.979426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bebeb-aaf0-4279-90f9-396ea4dab547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:55.981683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bebeb-aaf0-4279-90f9-396ea4dab547 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:56.015639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1003/1971] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:55 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:56.020216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4eec8e-ade2-497c-95c1-a15042553344 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:56.037483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4eec8e-ade2-497c-95c1-a15042553344 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:56.106651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499ffcb-bd0c-4c31-a1f2-2c5d24d3b9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:56.107337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499ffcb-bd0c-4c31-a1f2-2c5d24d3b9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:56.107621 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499ffcb-bd0c-4c31-a1f2-2c5d24d3b9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:56.107896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499ffcb-bd0c-4c31-a1f2-2c5d24d3b9e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:56.108596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499ffcb-bd0c-4c31-a1f2-2c5d24d3b9e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:56.110578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499ffcb-bd0c-4c31-a1f2-2c5d24d3b9e4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:56.135044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 969/1972] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:56 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:56.139247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bebeb-aaf0-4279-90f9-396ea4dab547 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:56.151360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bebeb-aaf0-4279-90f9-396ea4dab547 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:56.156244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0498a1-a87d-41df-96f8-ee87d3e636bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:56.156927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0498a1-a87d-41df-96f8-ee87d3e636bb admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:56.157206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0498a1-a87d-41df-96f8-ee87d3e636bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:56.157479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0498a1-a87d-41df-96f8-ee87d3e636bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:56.158251 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0498a1-a87d-41df-96f8-ee87d3e636bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:56.159923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0498a1-a87d-41df-96f8-ee87d3e636bb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:56.172610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1004/1973] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:56.176964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499ffcb-bd0c-4c31-a1f2-2c5d24d3b9e4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:56.187936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499ffcb-bd0c-4c31-a1f2-2c5d24d3b9e4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:56.192950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8fdcd-2da1-4c94-a5b1-a2b95f286e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:56.193643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8fdcd-2da1-4c94-a5b1-a2b95f286e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:56.194513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8fdcd-2da1-4c94-a5b1-a2b95f286e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:56.194837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8fdcd-2da1-4c94-a5b1-a2b95f286e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:56.195560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8fdcd-2da1-4c94-a5b1-a2b95f286e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:56.197228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8fdcd-2da1-4c94-a5b1-a2b95f286e27 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:56.209068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 970/1974] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:56.217189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0498a1-a87d-41df-96f8-ee87d3e636bb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:56.229224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0498a1-a87d-41df-96f8-ee87d3e636bb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:56.234694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce563ab-0633-421a-bd50-ae2ae60e474c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:56.235617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce563ab-0633-421a-bd50-ae2ae60e474c admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:56.235907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce563ab-0633-421a-bd50-ae2ae60e474c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:56.236183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce563ab-0633-421a-bd50-ae2ae60e474c admin admin] PATH_INFO: `/v3/projects/4482f8d3cf8049ea81664d0a55f62cb8/users/e706d92ac8c44bf1a21c4e0fa7ed1ecc/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:56.255749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce563ab-0633-421a-bd50-ae2ae60e474c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4482f8d3cf8049ea81664d0a55f62cb8, user_id=e706d92ac8c44bf1a21c4e0fa7ed1ecc, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:56.261595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce563ab-0633-421a-bd50-ae2ae60e474c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:56.281691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1005/1975] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:56 2022] PUT /identity/v3/projects/4482f8d3cf8049ea81664d0a55f62cb8/users/e706d92ac8c44bf1a21c4e0fa7ed1ecc/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:56.296464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18d1b3-2c58-4037-9eb8-592503271d47 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:56.296888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18d1b3-2c58-4037-9eb8-592503271d47 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:56.297289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18d1b3-2c58-4037-9eb8-592503271d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:56.314127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:56.334041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fe18d1b3-2c58-4037-9eb8-592503271d47 None None] MFA Rules not processed for user `e706d92ac8c44bf1a21c4e0fa7ed1ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:56.336115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fe18d1b3-2c58-4037-9eb8-592503271d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:56.423520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 971/1976] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:56.429060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce563ab-0633-421a-bd50-ae2ae60e474c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:56.505415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce563ab-0633-421a-bd50-ae2ae60e474c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:56.513003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e919fc-4b0e-48f0-94ec-57749e6cdb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:56.513787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e919fc-4b0e-48f0-94ec-57749e6cdb8d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:56.514140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e919fc-4b0e-48f0-94ec-57749e6cdb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:56.514457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e919fc-4b0e-48f0-94ec-57749e6cdb8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:56.515211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e919fc-4b0e-48f0-94ec-57749e6cdb8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:56.517275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e919fc-4b0e-48f0-94ec-57749e6cdb8d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:56.520994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:56.556120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1006/1977] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:23:56 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:56.562823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18d1b3-2c58-4037-9eb8-592503271d47 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:56.582620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18d1b3-2c58-4037-9eb8-592503271d47 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:56.681864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c657c-9ea6-41b5-b12e-880782eb9cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:56.683039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c657c-9ea6-41b5-b12e-880782eb9cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:56.683569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c657c-9ea6-41b5-b12e-880782eb9cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:56.684302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c657c-9ea6-41b5-b12e-880782eb9cdc admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:56.685611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c657c-9ea6-41b5-b12e-880782eb9cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:56.689534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c657c-9ea6-41b5-b12e-880782eb9cdc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:56.730434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 972/1978] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:23:56 2022] POST /identity/v3/users => generated 306 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:23:56.735930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e919fc-4b0e-48f0-94ec-57749e6cdb8d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:56.752469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e919fc-4b0e-48f0-94ec-57749e6cdb8d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:56.758710 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b183a-d75e-413b-b643-4faff82ae79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:56.759871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b183a-d75e-413b-b643-4faff82ae79b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:56.760372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b183a-d75e-413b-b643-4faff82ae79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:56.760861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b183a-d75e-413b-b643-4faff82ae79b admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:56.762200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b183a-d75e-413b-b643-4faff82ae79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:56.765202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b183a-d75e-413b-b643-4faff82ae79b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:56.792022 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1007/1979] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:23:56.795464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c657c-9ea6-41b5-b12e-880782eb9cdc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:56.807995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c657c-9ea6-41b5-b12e-880782eb9cdc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:56.813106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe76c94-93fb-4340-bf59-202a979fe8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:56.814390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe76c94-93fb-4340-bf59-202a979fe8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:56.814585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe76c94-93fb-4340-bf59-202a979fe8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:56.814898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe76c94-93fb-4340-bf59-202a979fe8fe admin admin] PATH_INFO: `/v3/projects/8c48665b4a6c44d99db9e12abd816794/users/42f84fc8250747a6aba07401b37b0520/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:56.839657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe76c94-93fb-4340-bf59-202a979fe8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c48665b4a6c44d99db9e12abd816794, user_id=42f84fc8250747a6aba07401b37b0520, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:56.844089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe76c94-93fb-4340-bf59-202a979fe8fe admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:56.865461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 973/1980] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:56 2022] PUT /identity/v3/projects/8c48665b4a6c44d99db9e12abd816794/users/42f84fc8250747a6aba07401b37b0520/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:23:56.872188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b183a-d75e-413b-b643-4faff82ae79b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:56.894550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b183a-d75e-413b-b643-4faff82ae79b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:57.059217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ab44-39f5-4b56-820c-924c171a86bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:57.059869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ab44-39f5-4b56-820c-924c171a86bc admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:57.060274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ab44-39f5-4b56-820c-924c171a86bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:57.060631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ab44-39f5-4b56-820c-924c171a86bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:57.061462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ab44-39f5-4b56-820c-924c171a86bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:57.063437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5ab44-39f5-4b56-820c-924c171a86bc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:57.088745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1008/1981] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:23:57.094661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe76c94-93fb-4340-bf59-202a979fe8fe admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:57.123708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe76c94-93fb-4340-bf59-202a979fe8fe admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:57.136196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737df47c-807a-467e-8e30-e8db815921f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:57.138304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737df47c-807a-467e-8e30-e8db815921f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:57.138900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737df47c-807a-467e-8e30-e8db815921f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:57.139475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737df47c-807a-467e-8e30-e8db815921f9 admin admin] PATH_INFO: `/v3/projects/8c48665b4a6c44d99db9e12abd816794/users/42f84fc8250747a6aba07401b37b0520/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:57.144012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737df47c-807a-467e-8e30-e8db815921f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c48665b4a6c44d99db9e12abd816794, user_id=42f84fc8250747a6aba07401b37b0520, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:57.151182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737df47c-807a-467e-8e30-e8db815921f9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:57.182233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 974/1982] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:23:57 2022] PUT /identity/v3/projects/8c48665b4a6c44d99db9e12abd816794/users/42f84fc8250747a6aba07401b37b0520/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:23:57.186333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79402883-2f40-40ee-b9dd-55f211c2332d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:57.186660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79402883-2f40-40ee-b9dd-55f211c2332d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:57.186903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79402883-2f40-40ee-b9dd-55f211c2332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:57.241897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-79402883-2f40-40ee-b9dd-55f211c2332d None None] MFA Rules not processed for user `42f84fc8250747a6aba07401b37b0520`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:23:57.243452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-79402883-2f40-40ee-b9dd-55f211c2332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:57.371127 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1009/1983] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:23:57 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:23:57.393358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737df47c-807a-467e-8e30-e8db815921f9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:57.395998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-737df47c-807a-467e-8e30-e8db815921f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:23:57.476802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:57.484667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737df47c-807a-467e-8e30-e8db815921f9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:57.491968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d748f7-e1c1-459e-85a0-e3f2296ac68f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:57.493056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d748f7-e1c1-459e-85a0-e3f2296ac68f service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:57.493575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d748f7-e1c1-459e-85a0-e3f2296ac68f service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:57.494117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d748f7-e1c1-459e-85a0-e3f2296ac68f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:57.509743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d748f7-e1c1-459e-85a0-e3f2296ac68f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:57.512593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d748f7-e1c1-459e-85a0-e3f2296ac68f service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:57.530547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 975/1984] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:57 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:23:57.651293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79402883-2f40-40ee-b9dd-55f211c2332d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:23:57.664553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79402883-2f40-40ee-b9dd-55f211c2332d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:23:57.670141 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513d8a9-7aac-4235-ba05-52ae9b1684e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:23:57.672004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513d8a9-7aac-4235-ba05-52ae9b1684e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:23:57.672562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513d8a9-7aac-4235-ba05-52ae9b1684e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:23:57.673037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513d8a9-7aac-4235-ba05-52ae9b1684e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:23:57.784233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:23:57.800399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513d8a9-7aac-4235-ba05-52ae9b1684e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:23:57.805077 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513d8a9-7aac-4235-ba05-52ae9b1684e7 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:23:57.824283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1010/1985] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:23:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:01.961755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d748f7-e1c1-459e-85a0-e3f2296ac68f service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:01.965288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c9d748f7-e1c1-459e-85a0-e3f2296ac68f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:02.107184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d748f7-e1c1-459e-85a0-e3f2296ac68f service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:02.108527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:02.109343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:02.109688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:02.111838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] PATH_INFO: `/v3/users/9ba53391e2cf4db68ca01913570b8f64` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:02.111838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba53391e2cf4db68ca01913570b8f64)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:02.113736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:02.218281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba53391e2cf4db68ca01913570b8f64'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:02.230134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba53391e2cf4db68ca01913570b8f64'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:02.231711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba53391e2cf4db68ca01913570b8f64'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:02.260164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba53391e2cf4db68ca01913570b8f64'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:02.261845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba53391e2cf4db68ca01913570b8f64'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:02.294888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 976/1986] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:01 2022] DELETE /identity/v3/users/9ba53391e2cf4db68ca01913570b8f64 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:02.301999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a006a-b998-47f4-be16-9acfe5801079 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:02.302711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a006a-b998-47f4-be16-9acfe5801079 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:02.304844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a006a-b998-47f4-be16-9acfe5801079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:02.338090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ec0a006a-b998-47f4-be16-9acfe5801079 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:02.339773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ec0a006a-b998-47f4-be16-9acfe5801079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:02.441073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1011/1987] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:02.452829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:02.454543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:02.539626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16043d60-08c9-4222-b540-cde6d4bf5982 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:02.543979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fccdf4-dc33-4bb5-92d5-00473061a90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:02.544799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fccdf4-dc33-4bb5-92d5-00473061a90f service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:02.545129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fccdf4-dc33-4bb5-92d5-00473061a90f service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:02.545477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fccdf4-dc33-4bb5-92d5-00473061a90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:02.560739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fccdf4-dc33-4bb5-92d5-00473061a90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:02.562232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fccdf4-dc33-4bb5-92d5-00473061a90f service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:02.574767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 977/1988] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:02.932387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a006a-b998-47f4-be16-9acfe5801079 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:02.934330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ec0a006a-b998-47f4-be16-9acfe5801079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:02.950920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a006a-b998-47f4-be16-9acfe5801079 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:02.956027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:02.956914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:02.957196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:02.957470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] PATH_INFO: `/v3/projects/ffdc7ce35fdd4310b3c1c1f19f01574a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:02.958846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffdc7ce35fdd4310b3c1c1f19f01574a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:02.961954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:03.021421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffdc7ce35fdd4310b3c1c1f19f01574a', 'request_id': 'req-13d88b80-4b93-4086-8be6-5c6b17789428'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:03.031624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffdc7ce35fdd4310b3c1c1f19f01574a', 'request_id': 'req-13d88b80-4b93-4086-8be6-5c6b17789428'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:03.041756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffdc7ce35fdd4310b3c1c1f19f01574a', 'request_id': 'req-13d88b80-4b93-4086-8be6-5c6b17789428'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:03.057689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] The token cache is being invalidate because project ffdc7ce35fdd4310b3c1c1f19f01574a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:03.058154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:03.058911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-13d88b80-4b93-4086-8be6-5c6b17789428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:03.060646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1012/1989] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:02 2022] DELETE /identity/v3/projects/ffdc7ce35fdd4310b3c1c1f19f01574a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:03.066851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfb2a8-2cde-4618-96bb-e8963a2b7ed5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:03.067392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfb2a8-2cde-4618-96bb-e8963a2b7ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:03.067923 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfb2a8-2cde-4618-96bb-e8963a2b7ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:03.099733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b2cfb2a8-2cde-4618-96bb-e8963a2b7ed5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:03.102439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b2cfb2a8-2cde-4618-96bb-e8963a2b7ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:03.121556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:03.222074 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 978/1990] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:03.231727 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c1375-ec2f-40a6-a1ed-5a500241948a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:03.232039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c1375-ec2f-40a6-a1ed-5a500241948a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:03.232323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c1375-ec2f-40a6-a1ed-5a500241948a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:03.256247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-448c1375-ec2f-40a6-a1ed-5a500241948a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:03.258275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-448c1375-ec2f-40a6-a1ed-5a500241948a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:03.272537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1013/1991] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:03.277931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfb2a8-2cde-4618-96bb-e8963a2b7ed5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:03.295161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfb2a8-2cde-4618-96bb-e8963a2b7ed5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:03.302867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc87c8-e2bf-4542-a815-77cbcb3d4b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:03.303711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc87c8-e2bf-4542-a815-77cbcb3d4b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:03.304179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc87c8-e2bf-4542-a815-77cbcb3d4b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:03.304716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc87c8-e2bf-4542-a815-77cbcb3d4b76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:03.306314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc87c8-e2bf-4542-a815-77cbcb3d4b76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:24:03.306878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc87c8-e2bf-4542-a815-77cbcb3d4b76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:03.309798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc87c8-e2bf-4542-a815-77cbcb3d4b76 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:03.335937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 979/1992] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:24:03.340874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c1375-ec2f-40a6-a1ed-5a500241948a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:03.359290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c1375-ec2f-40a6-a1ed-5a500241948a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:03.369614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e72d3-128c-4950-95c4-9f14fcdd462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:03.370868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e72d3-128c-4950-95c4-9f14fcdd462f admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:03.371530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e72d3-128c-4950-95c4-9f14fcdd462f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:03.371907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e72d3-128c-4950-95c4-9f14fcdd462f admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:03.372756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e72d3-128c-4950-95c4-9f14fcdd462f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:03.376107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e72d3-128c-4950-95c4-9f14fcdd462f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:03.466808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1014/1993] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:03 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:24:03.471094 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc87c8-e2bf-4542-a815-77cbcb3d4b76 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:03.494762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc87c8-e2bf-4542-a815-77cbcb3d4b76 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:03.643831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15aa1e-afbf-4be8-a775-fde919e53184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:03.644692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15aa1e-afbf-4be8-a775-fde919e53184 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:03.645120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15aa1e-afbf-4be8-a775-fde919e53184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:03.645690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15aa1e-afbf-4be8-a775-fde919e53184 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:03.646969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15aa1e-afbf-4be8-a775-fde919e53184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:03.650199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15aa1e-afbf-4be8-a775-fde919e53184 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:03.698076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 980/1994] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:03 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:03.702867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e72d3-128c-4950-95c4-9f14fcdd462f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:03.723309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e72d3-128c-4950-95c4-9f14fcdd462f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:03.731975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197a4e-3fb5-4f92-9cba-71e10cad8c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:03.732817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197a4e-3fb5-4f92-9cba-71e10cad8c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:03.733472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197a4e-3fb5-4f92-9cba-71e10cad8c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:03.733791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197a4e-3fb5-4f92-9cba-71e10cad8c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:03.734974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a197a4e-3fb5-4f92-9cba-71e10cad8c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:03.736956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a197a4e-3fb5-4f92-9cba-71e10cad8c3f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:03.758798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1015/1995] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:24:03.763193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15aa1e-afbf-4be8-a775-fde919e53184 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:03.781129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15aa1e-afbf-4be8-a775-fde919e53184 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:03.793100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec1250-189b-4006-891b-c53177c9204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:03.794595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec1250-189b-4006-891b-c53177c9204c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:03.795450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec1250-189b-4006-891b-c53177c9204c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:03.795855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec1250-189b-4006-891b-c53177c9204c admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:03.796898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec1250-189b-4006-891b-c53177c9204c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:03.799393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec1250-189b-4006-891b-c53177c9204c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:03.815818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 981/1996] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:24:03.821974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197a4e-3fb5-4f92-9cba-71e10cad8c3f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:03.842880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197a4e-3fb5-4f92-9cba-71e10cad8c3f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:03.856626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6d823-a155-4bb7-976b-4429d913ac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:03.858316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6d823-a155-4bb7-976b-4429d913ac16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:03.858893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6d823-a155-4bb7-976b-4429d913ac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:03.859325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6d823-a155-4bb7-976b-4429d913ac16 admin admin] PATH_INFO: `/v3/projects/b3f0dac2d1084c728b0d534da54817da/users/fa06b07c4f324459a35a2dcce1c6b4c8/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:03.885671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6d823-a155-4bb7-976b-4429d913ac16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f0dac2d1084c728b0d534da54817da, user_id=fa06b07c4f324459a35a2dcce1c6b4c8, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:03.888803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6d823-a155-4bb7-976b-4429d913ac16 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:03.913453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1016/1997] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:03 2022] PUT /identity/v3/projects/b3f0dac2d1084c728b0d534da54817da/users/fa06b07c4f324459a35a2dcce1c6b4c8/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:03.928710 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa34a66-7739-4414-9137-22fb36cfd72f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:03.929361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa34a66-7739-4414-9137-22fb36cfd72f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:03.929948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa34a66-7739-4414-9137-22fb36cfd72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:03.979284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8fa34a66-7739-4414-9137-22fb36cfd72f None None] MFA Rules not processed for user `fa06b07c4f324459a35a2dcce1c6b4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:03.982697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8fa34a66-7739-4414-9137-22fb36cfd72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:04.112221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 982/1998] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:04.119777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6d823-a155-4bb7-976b-4429d913ac16 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:04.153336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:04.276469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6d823-a155-4bb7-976b-4429d913ac16 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:04.285593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f31126-5a16-498a-85b1-5f007a0d252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:04.286895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f31126-5a16-498a-85b1-5f007a0d252e admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:04.288038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f31126-5a16-498a-85b1-5f007a0d252e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:04.288602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f31126-5a16-498a-85b1-5f007a0d252e admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:04.290027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f31126-5a16-498a-85b1-5f007a0d252e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:04.293750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f31126-5a16-498a-85b1-5f007a0d252e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:04.331496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1017/1999] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:04 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:24:04.337039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa34a66-7739-4414-9137-22fb36cfd72f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:04.355409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa34a66-7739-4414-9137-22fb36cfd72f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:04.362077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:04.443202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec18f65-b903-4663-b9a6-f6b0b7f62fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:04.444504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec18f65-b903-4663-b9a6-f6b0b7f62fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:04.444819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec18f65-b903-4663-b9a6-f6b0b7f62fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:04.445284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec18f65-b903-4663-b9a6-f6b0b7f62fe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:04.446067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec18f65-b903-4663-b9a6-f6b0b7f62fe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:04.448083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec18f65-b903-4663-b9a6-f6b0b7f62fe3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:04.484905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 983/2000] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:04 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:24:04.491462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f31126-5a16-498a-85b1-5f007a0d252e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:04.515782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f31126-5a16-498a-85b1-5f007a0d252e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:04.528067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69adac7-e4ee-4ecc-8e51-3b78b15b48af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:04.529528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69adac7-e4ee-4ecc-8e51-3b78b15b48af admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:04.530340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69adac7-e4ee-4ecc-8e51-3b78b15b48af admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:04.530612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69adac7-e4ee-4ecc-8e51-3b78b15b48af admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:04.532090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69adac7-e4ee-4ecc-8e51-3b78b15b48af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:04.534169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69adac7-e4ee-4ecc-8e51-3b78b15b48af admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:04.565984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1018/2001] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:04.574077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec18f65-b903-4663-b9a6-f6b0b7f62fe3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:04.593533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec18f65-b903-4663-b9a6-f6b0b7f62fe3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:04.610817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7716f5-57ff-4f05-87ab-b97df62828a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:04.612140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7716f5-57ff-4f05-87ab-b97df62828a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:04.612562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7716f5-57ff-4f05-87ab-b97df62828a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:04.613077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7716f5-57ff-4f05-87ab-b97df62828a5 admin admin] PATH_INFO: `/v3/projects/d6e135dc715040f6a0706650d7cbbc8d/users/b846c4733e404fb7b6b84dd979854ed2/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:04.652649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7716f5-57ff-4f05-87ab-b97df62828a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e135dc715040f6a0706650d7cbbc8d, user_id=b846c4733e404fb7b6b84dd979854ed2, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:04.658484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7716f5-57ff-4f05-87ab-b97df62828a5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:04.686920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 984/2002] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:04 2022] PUT /identity/v3/projects/d6e135dc715040f6a0706650d7cbbc8d/users/b846c4733e404fb7b6b84dd979854ed2/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:04.692444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69adac7-e4ee-4ecc-8e51-3b78b15b48af admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:04.704964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69adac7-e4ee-4ecc-8e51-3b78b15b48af admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:04.772230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7716f5-57ff-4f05-87ab-b97df62828a5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:04.777616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6a7716f5-57ff-4f05-87ab-b97df62828a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:04.787318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b187-a4db-4c36-a054-9143df2df7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:04.788222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b187-a4db-4c36-a054-9143df2df7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:04.788662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b187-a4db-4c36-a054-9143df2df7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:04.789162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b187-a4db-4c36-a054-9143df2df7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:04.790205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8b187-a4db-4c36-a054-9143df2df7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:04.792480 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8b187-a4db-4c36-a054-9143df2df7f0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:04.808687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1019/2003] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:04.811081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7716f5-57ff-4f05-87ab-b97df62828a5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:04.815244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b187-a4db-4c36-a054-9143df2df7f0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:04.818153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:04.819610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:04.819899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:04.820175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] PATH_INFO: `/v3/users/e706d92ac8c44bf1a21c4e0fa7ed1ecc` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:04.821526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e706d92ac8c44bf1a21c4e0fa7ed1ecc)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:04.825628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:04.833474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b187-a4db-4c36-a054-9143df2df7f0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:04.839747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147947a6-8e3f-437a-a777-247ae9d74e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:04.840785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147947a6-8e3f-437a-a777-247ae9d74e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:04.841167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147947a6-8e3f-437a-a777-247ae9d74e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:04.841623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147947a6-8e3f-437a-a777-247ae9d74e38 admin admin] PATH_INFO: `/v3/projects/d6e135dc715040f6a0706650d7cbbc8d/users/b846c4733e404fb7b6b84dd979854ed2/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:04.844402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147947a6-8e3f-437a-a777-247ae9d74e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e135dc715040f6a0706650d7cbbc8d, user_id=b846c4733e404fb7b6b84dd979854ed2, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:04.847470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147947a6-8e3f-437a-a777-247ae9d74e38 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:04.861459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1020/2004] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:04 2022] PUT /identity/v3/projects/d6e135dc715040f6a0706650d7cbbc8d/users/b846c4733e404fb7b6b84dd979854ed2/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:04.873133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a03d8c-3f17-48ca-b20e-7f0077b9d87d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:04.873481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a03d8c-3f17-48ca-b20e-7f0077b9d87d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:04.873898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a03d8c-3f17-48ca-b20e-7f0077b9d87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:04.924144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d5a03d8c-3f17-48ca-b20e-7f0077b9d87d None None] MFA Rules not processed for user `b846c4733e404fb7b6b84dd979854ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:04.927159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d5a03d8c-3f17-48ca-b20e-7f0077b9d87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:04.933301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e706d92ac8c44bf1a21c4e0fa7ed1ecc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:04.941985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e706d92ac8c44bf1a21c4e0fa7ed1ecc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:04.942797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e706d92ac8c44bf1a21c4e0fa7ed1ecc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:04.956594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e706d92ac8c44bf1a21c4e0fa7ed1ecc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:04.957345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba0fdcb8-f0c1-46a9-a067-341e138817d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e706d92ac8c44bf1a21c4e0fa7ed1ecc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:04.980433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 985/2005] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:04 2022] DELETE /identity/v3/users/e706d92ac8c44bf1a21c4e0fa7ed1ecc => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:04.987093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22094d15-2515-47ce-8219-918bd78eb52d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:04.987670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22094d15-2515-47ce-8219-918bd78eb52d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:04.988209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22094d15-2515-47ce-8219-918bd78eb52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:05.020433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-22094d15-2515-47ce-8219-918bd78eb52d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:05.023787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-22094d15-2515-47ce-8219-918bd78eb52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:05.040786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:05.065460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1021/2006] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:24:05.074947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a03d8c-3f17-48ca-b20e-7f0077b9d87d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:05.076677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d5a03d8c-3f17-48ca-b20e-7f0077b9d87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:05.095900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:05.137679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 986/2007] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:05.148944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22094d15-2515-47ce-8219-918bd78eb52d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:05.151029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-22094d15-2515-47ce-8219-918bd78eb52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:05.172502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a03d8c-3f17-48ca-b20e-7f0077b9d87d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:05.245016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22094d15-2515-47ce-8219-918bd78eb52d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:05.250156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5563c-e04a-4ffb-936a-67e35984314c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:05.250927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5563c-e04a-4ffb-936a-67e35984314c service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:05.251400 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5563c-e04a-4ffb-936a-67e35984314c service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:05.251752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5563c-e04a-4ffb-936a-67e35984314c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:05.265018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3f7af-d35e-4488-8904-ba7764e7a3e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:05.265735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3f7af-d35e-4488-8904-ba7764e7a3e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:05.266031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3f7af-d35e-4488-8904-ba7764e7a3e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:05.266429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3f7af-d35e-4488-8904-ba7764e7a3e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:05.268420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3fa3f7af-d35e-4488-8904-ba7764e7a3e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:05.276211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5563c-e04a-4ffb-936a-67e35984314c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:05.279056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5563c-e04a-4ffb-936a-67e35984314c service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:05.298084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 987/2008] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:24:05.370261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3f7af-d35e-4488-8904-ba7764e7a3e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:05.371793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3f7af-d35e-4488-8904-ba7764e7a3e1 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:05.393520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1022/2009] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:24:05.709014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5563c-e04a-4ffb-936a-67e35984314c service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:05.711749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-52a5563c-e04a-4ffb-936a-67e35984314c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:05.755484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5563c-e04a-4ffb-936a-67e35984314c service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:05.763602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:05.764935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:05.765412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:05.765991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] PATH_INFO: `/v3/projects/4482f8d3cf8049ea81664d0a55f62cb8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:05.767976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=4482f8d3cf8049ea81664d0a55f62cb8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:05.770526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:05.876020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4482f8d3cf8049ea81664d0a55f62cb8', 'request_id': 'req-f4da5bb1-6534-41a0-99b1-8662688a34ab'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:05.896284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4482f8d3cf8049ea81664d0a55f62cb8', 'request_id': 'req-f4da5bb1-6534-41a0-99b1-8662688a34ab'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:05.905579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4482f8d3cf8049ea81664d0a55f62cb8', 'request_id': 'req-f4da5bb1-6534-41a0-99b1-8662688a34ab'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:05.925239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] The token cache is being invalidate because project 4482f8d3cf8049ea81664d0a55f62cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:05.925765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:05.927190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:05.934119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 988/2010] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:05 2022] DELETE /identity/v3/projects/4482f8d3cf8049ea81664d0a55f62cb8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:05.938560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3f7af-d35e-4488-8904-ba7764e7a3e1 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:05.962093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3f7af-d35e-4488-8904-ba7764e7a3e1 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:06.105509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:06.106611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:06.107088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:06.107386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] PATH_INFO: `/v3/users/42f84fc8250747a6aba07401b37b0520` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:06.108749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f84fc8250747a6aba07401b37b0520)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:06.110824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:06.120783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:06.186093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f84fc8250747a6aba07401b37b0520'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:06.195868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f84fc8250747a6aba07401b37b0520'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:06.196724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f84fc8250747a6aba07401b37b0520'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:06.216621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f84fc8250747a6aba07401b37b0520'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:06.241983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7aba28c0-52a8-4878-9e7b-d876467b841f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f84fc8250747a6aba07401b37b0520'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:06.249357 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1023/2011] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:05 2022] DELETE /identity/v3/users/42f84fc8250747a6aba07401b37b0520 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:06.678968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:06.681549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:06.825222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da5bb1-6534-41a0-99b1-8662688a34ab admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:06.886869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:06.952480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:06.954232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:06.954920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:06.955357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] PATH_INFO: `/v3/projects/8c48665b4a6c44d99db9e12abd816794` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:06.957818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c48665b4a6c44d99db9e12abd816794)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:06.966674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:07.102955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c48665b4a6c44d99db9e12abd816794', 'request_id': 'req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:07.121684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c48665b4a6c44d99db9e12abd816794', 'request_id': 'req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:07.136481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c48665b4a6c44d99db9e12abd816794', 'request_id': 'req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:07.174011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] The token cache is being invalidate because project 8c48665b4a6c44d99db9e12abd816794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:07.174678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:07.175829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8bab4b00-66fe-4a2b-8eca-cdfcab3876a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:07.186168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 989/2012] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:06 2022] DELETE /identity/v3/projects/8c48665b4a6c44d99db9e12abd816794 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:07.188575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bf469-4dd5-4e09-a5b8-005559e2e4a1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:07.189304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bf469-4dd5-4e09-a5b8-005559e2e4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:07.189640 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bf469-4dd5-4e09-a5b8-005559e2e4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:07.289731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-eb7bf469-4dd5-4e09-a5b8-005559e2e4a1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:07.291925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-eb7bf469-4dd5-4e09-a5b8-005559e2e4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:07.444608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1024/2013] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:07.458050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fc0d3-29cf-495c-9649-cd27602fc965 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:07.458641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fc0d3-29cf-495c-9649-cd27602fc965 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:07.459088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fc0d3-29cf-495c-9649-cd27602fc965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:07.490395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bf3fc0d3-29cf-495c-9649-cd27602fc965 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:07.492776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bf3fc0d3-29cf-495c-9649-cd27602fc965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:07.521756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 990/2014] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:24:07.525693 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bf469-4dd5-4e09-a5b8-005559e2e4a1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:07.536308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bf469-4dd5-4e09-a5b8-005559e2e4a1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:07.542577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f240ec-29a1-48f9-b78f-b4c4634a4a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:07.543103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f240ec-29a1-48f9-b78f-b4c4634a4a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:07.543470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f240ec-29a1-48f9-b78f-b4c4634a4a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:07.543768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f240ec-29a1-48f9-b78f-b4c4634a4a79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:07.544593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f240ec-29a1-48f9-b78f-b4c4634a4a79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:24:07.544910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f240ec-29a1-48f9-b78f-b4c4634a4a79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:07.547102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f240ec-29a1-48f9-b78f-b4c4634a4a79 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:07.565349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1025/2015] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:24:07.572247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3fc0d3-29cf-495c-9649-cd27602fc965 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:07.584303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3fc0d3-29cf-495c-9649-cd27602fc965 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:07.590569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b869b4-9040-4dd1-8f13-70fa59998bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:07.591345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b869b4-9040-4dd1-8f13-70fa59998bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:07.591596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b869b4-9040-4dd1-8f13-70fa59998bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:07.591877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b869b4-9040-4dd1-8f13-70fa59998bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:07.592665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b869b4-9040-4dd1-8f13-70fa59998bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:07.595043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b869b4-9040-4dd1-8f13-70fa59998bbe admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:07.627947 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 991/2016] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:07 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:07.633033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f240ec-29a1-48f9-b78f-b4c4634a4a79 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:07.644265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f240ec-29a1-48f9-b78f-b4c4634a4a79 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:07.732963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a421c7e-071d-47ab-8e51-f4ff4b9d8412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:07.733583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a421c7e-071d-47ab-8e51-f4ff4b9d8412 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:07.733920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a421c7e-071d-47ab-8e51-f4ff4b9d8412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:07.734202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a421c7e-071d-47ab-8e51-f4ff4b9d8412 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:07.734927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a421c7e-071d-47ab-8e51-f4ff4b9d8412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:07.736903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a421c7e-071d-47ab-8e51-f4ff4b9d8412 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:07.749617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:07.769812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1026/2017] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:07 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:07.775238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b869b4-9040-4dd1-8f13-70fa59998bbe admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:07.791229 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b869b4-9040-4dd1-8f13-70fa59998bbe admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:07.801946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e57372-548c-4435-a3be-6cdeaaf0db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:07.803729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57372-548c-4435-a3be-6cdeaaf0db5b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:07.804213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57372-548c-4435-a3be-6cdeaaf0db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:07.804712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57372-548c-4435-a3be-6cdeaaf0db5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:07.806076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e57372-548c-4435-a3be-6cdeaaf0db5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:07.809202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e57372-548c-4435-a3be-6cdeaaf0db5b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:07.823907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 992/2018] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:07.828972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a421c7e-071d-47ab-8e51-f4ff4b9d8412 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:07.842216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a421c7e-071d-47ab-8e51-f4ff4b9d8412 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:07.850564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067807a-622e-4216-a15a-9344c951443d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:07.851262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067807a-622e-4216-a15a-9344c951443d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:07.851552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067807a-622e-4216-a15a-9344c951443d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:07.851833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067807a-622e-4216-a15a-9344c951443d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:07.852589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067807a-622e-4216-a15a-9344c951443d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:07.855196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067807a-622e-4216-a15a-9344c951443d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:07.867583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1027/2019] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:07.875459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e57372-548c-4435-a3be-6cdeaaf0db5b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:07.893529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e57372-548c-4435-a3be-6cdeaaf0db5b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:07.900686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c17df7-c0b3-471e-a281-74519c944b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:07.901753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c17df7-c0b3-471e-a281-74519c944b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:07.902046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c17df7-c0b3-471e-a281-74519c944b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:07.902408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c17df7-c0b3-471e-a281-74519c944b72 admin admin] PATH_INFO: `/v3/projects/810c087aef3d453cb9c0365467dd9134/users/66ded9372b644c57a9f13fc88ff7fdb6/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:07.919425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c17df7-c0b3-471e-a281-74519c944b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=810c087aef3d453cb9c0365467dd9134, user_id=66ded9372b644c57a9f13fc88ff7fdb6, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:07.922157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c17df7-c0b3-471e-a281-74519c944b72 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:07.937430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 993/2020] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:07 2022] PUT /identity/v3/projects/810c087aef3d453cb9c0365467dd9134/users/66ded9372b644c57a9f13fc88ff7fdb6/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:07.942739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355aa34-bb51-4d60-92f9-77fe402670e1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:07.943032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355aa34-bb51-4d60-92f9-77fe402670e1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:07.943325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355aa34-bb51-4d60-92f9-77fe402670e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:07.969298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a355aa34-bb51-4d60-92f9-77fe402670e1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:07.971546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a355aa34-bb51-4d60-92f9-77fe402670e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:08.056834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1028/2021] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:08.066870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c17df7-c0b3-471e-a281-74519c944b72 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:08.069332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-10c17df7-c0b3-471e-a281-74519c944b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:08.169278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:08.232920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c17df7-c0b3-471e-a281-74519c944b72 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:08.241664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404beec4-1ea7-4049-baf3-1985d218c32c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:08.243321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404beec4-1ea7-4049-baf3-1985d218c32c service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:08.243805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404beec4-1ea7-4049-baf3-1985d218c32c service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:08.244304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404beec4-1ea7-4049-baf3-1985d218c32c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:08.260328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404beec4-1ea7-4049-baf3-1985d218c32c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:08.261816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404beec4-1ea7-4049-baf3-1985d218c32c service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:08.274561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 994/2022] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:24:08.519969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355aa34-bb51-4d60-92f9-77fe402670e1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:08.525246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a355aa34-bb51-4d60-92f9-77fe402670e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:08.559265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355aa34-bb51-4d60-92f9-77fe402670e1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:08.566495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:08.567909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:08.570667 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:08.571133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] PATH_INFO: `/v3/users/fa06b07c4f324459a35a2dcce1c6b4c8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:08.573104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa06b07c4f324459a35a2dcce1c6b4c8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:08.575988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:08.675184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa06b07c4f324459a35a2dcce1c6b4c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:08.696694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa06b07c4f324459a35a2dcce1c6b4c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:08.710965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa06b07c4f324459a35a2dcce1c6b4c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:08.731734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa06b07c4f324459a35a2dcce1c6b4c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:08.748327 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa06b07c4f324459a35a2dcce1c6b4c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:08.757385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1029/2023] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:08 2022] DELETE /identity/v3/users/fa06b07c4f324459a35a2dcce1c6b4c8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:08.764601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8d555-1ea7-4f2c-b3df-3afcd984d96b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:08.765136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8d555-1ea7-4f2c-b3df-3afcd984d96b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:08.765634 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8d555-1ea7-4f2c-b3df-3afcd984d96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:08.805058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-38b8d555-1ea7-4f2c-b3df-3afcd984d96b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:08.807194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-38b8d555-1ea7-4f2c-b3df-3afcd984d96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:09.018759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 995/2024] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:09.036956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:09.046549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:09.160789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dca32d-483d-434e-af11-d954e73498b9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:09.169142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4073c44-97de-4947-b63b-1645cff263be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:09.170336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4073c44-97de-4947-b63b-1645cff263be service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:09.170842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4073c44-97de-4947-b63b-1645cff263be service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:09.171342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4073c44-97de-4947-b63b-1645cff263be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:09.185170 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:09.192666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4073c44-97de-4947-b63b-1645cff263be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:09.196044 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4073c44-97de-4947-b63b-1645cff263be service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:09.218135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1030/2025] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:09.487686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5002f-a5cc-4f7e-a144-517374c651b6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:09.487992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5002f-a5cc-4f7e-a144-517374c651b6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:09.488295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5002f-a5cc-4f7e-a144-517374c651b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:09.531221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a9d5002f-a5cc-4f7e-a144-517374c651b6 None None] MFA Rules not processed for user `66ded9372b644c57a9f13fc88ff7fdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:09.533740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a9d5002f-a5cc-4f7e-a144-517374c651b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:09.687313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4073c44-97de-4947-b63b-1645cff263be service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:09.689819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b4073c44-97de-4947-b63b-1645cff263be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:09.707674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4073c44-97de-4947-b63b-1645cff263be service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:09.714712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:09.716835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:09.716835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:09.717145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] PATH_INFO: `/v3/projects/b3f0dac2d1084c728b0d534da54817da` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:09.718623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f0dac2d1084c728b0d534da54817da)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:09.720748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:09.760354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 996/2026] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:24:09.776632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5002f-a5cc-4f7e-a144-517374c651b6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:09.781208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a9d5002f-a5cc-4f7e-a144-517374c651b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:09.822832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f0dac2d1084c728b0d534da54817da', 'request_id': 'req-7fe66e4c-d41e-46be-9d65-b12295d77307'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:09.850433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f0dac2d1084c728b0d534da54817da', 'request_id': 'req-7fe66e4c-d41e-46be-9d65-b12295d77307'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:09.853681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f0dac2d1084c728b0d534da54817da', 'request_id': 'req-7fe66e4c-d41e-46be-9d65-b12295d77307'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:09.862952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5002f-a5cc-4f7e-a144-517374c651b6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:09.868058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] The token cache is being invalidate because project b3f0dac2d1084c728b0d534da54817da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:09.868345 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:09.869269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:09.869453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:09.872011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1031/2027] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:09 2022] DELETE /identity/v3/projects/b3f0dac2d1084c728b0d534da54817da => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:09.876228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:09.882912 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:09.930193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37b1bc-4016-45b9-90fa-ba7dd9d1347a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:09.930993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37b1bc-4016-45b9-90fa-ba7dd9d1347a service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:09.931353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37b1bc-4016-45b9-90fa-ba7dd9d1347a service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:09.931702 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37b1bc-4016-45b9-90fa-ba7dd9d1347a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:09.933839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5c37b1bc-4016-45b9-90fa-ba7dd9d1347a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:09.984909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe66e4c-d41e-46be-9d65-b12295d77307 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:09.996265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:09.998658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:09.999164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:09.999641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] PATH_INFO: `/v3/users/b846c4733e404fb7b6b84dd979854ed2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:10.005063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b846c4733e404fb7b6b84dd979854ed2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:10.009605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:10.041242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37b1bc-4016-45b9-90fa-ba7dd9d1347a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:10.044402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37b1bc-4016-45b9-90fa-ba7dd9d1347a service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:10.070475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 997/2028] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:10.137913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b846c4733e404fb7b6b84dd979854ed2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:10.156496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b846c4733e404fb7b6b84dd979854ed2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:10.157826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b846c4733e404fb7b6b84dd979854ed2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:10.177314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b846c4733e404fb7b6b84dd979854ed2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:10.219343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:10.221313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba8c5012-448e-466c-88cf-abf90f80e19a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b846c4733e404fb7b6b84dd979854ed2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:10.237368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1032/2029] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:09 2022] DELETE /identity/v3/users/b846c4733e404fb7b6b84dd979854ed2 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:10.837674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37b1bc-4016-45b9-90fa-ba7dd9d1347a service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:10.860458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37b1bc-4016-45b9-90fa-ba7dd9d1347a service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:10.981665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:10.982981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:10.983394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:10.983789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] PATH_INFO: `/v3/projects/d6e135dc715040f6a0706650d7cbbc8d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:10.985753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e135dc715040f6a0706650d7cbbc8d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:10.987939 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:11.063299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e135dc715040f6a0706650d7cbbc8d', 'request_id': 'req-9725f428-24b6-47b7-8699-edade19d3d72'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:11.072633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e135dc715040f6a0706650d7cbbc8d', 'request_id': 'req-9725f428-24b6-47b7-8699-edade19d3d72'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:11.086867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e135dc715040f6a0706650d7cbbc8d', 'request_id': 'req-9725f428-24b6-47b7-8699-edade19d3d72'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:11.091707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:11.110752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] The token cache is being invalidate because project d6e135dc715040f6a0706650d7cbbc8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:11.111047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:11.112043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9725f428-24b6-47b7-8699-edade19d3d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:11.114010 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 998/2030] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:10 2022] DELETE /identity/v3/projects/d6e135dc715040f6a0706650d7cbbc8d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:11.120333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055be511-eef0-45cb-be90-13a21002f025 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.121158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055be511-eef0-45cb-be90-13a21002f025 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.121158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055be511-eef0-45cb-be90-13a21002f025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.140925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-055be511-eef0-45cb-be90-13a21002f025 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:11.142209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-055be511-eef0-45cb-be90-13a21002f025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:11.240263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1033/2031] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:24:11.244328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b86ab-3d38-4602-90da-e98a8517b648 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.244653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b86ab-3d38-4602-90da-e98a8517b648 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.244930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b86ab-3d38-4602-90da-e98a8517b648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.268095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f21b86ab-3d38-4602-90da-e98a8517b648 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:11.269562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f21b86ab-3d38-4602-90da-e98a8517b648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:11.279483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 999/2032] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:11.284367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055be511-eef0-45cb-be90-13a21002f025 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:11.302125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055be511-eef0-45cb-be90-13a21002f025 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:11.308582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3f1e3-0ac8-4fb6-bf0c-1396cfe67701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:11.309276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3f1e3-0ac8-4fb6-bf0c-1396cfe67701 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.309561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3f1e3-0ac8-4fb6-bf0c-1396cfe67701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.309872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3f1e3-0ac8-4fb6-bf0c-1396cfe67701 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.310622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3f1e3-0ac8-4fb6-bf0c-1396cfe67701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:24:11.310931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3f1e3-0ac8-4fb6-bf0c-1396cfe67701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:11.312569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3f1e3-0ac8-4fb6-bf0c-1396cfe67701 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:11.327719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1034/2033] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:24:11.335443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b86ab-3d38-4602-90da-e98a8517b648 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:11.357683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b86ab-3d38-4602-90da-e98a8517b648 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:11.401244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a6ea4-cdec-4400-963a-db30f0cafc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:11.402401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a6ea4-cdec-4400-963a-db30f0cafc75 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.402893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a6ea4-cdec-4400-963a-db30f0cafc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.403381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a6ea4-cdec-4400-963a-db30f0cafc75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.404692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a6ea4-cdec-4400-963a-db30f0cafc75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:11.408432 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a6ea4-cdec-4400-963a-db30f0cafc75 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:11.471176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1000/2034] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:11 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:24:11.478508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3f1e3-0ac8-4fb6-bf0c-1396cfe67701 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:11.509275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3f1e3-0ac8-4fb6-bf0c-1396cfe67701 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:11.599739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586156a6-0d99-4dfa-8275-9a1a2121923c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:11.601828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586156a6-0d99-4dfa-8275-9a1a2121923c admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.602250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586156a6-0d99-4dfa-8275-9a1a2121923c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.602637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586156a6-0d99-4dfa-8275-9a1a2121923c admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.603876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586156a6-0d99-4dfa-8275-9a1a2121923c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:11.607702 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586156a6-0d99-4dfa-8275-9a1a2121923c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:11.643538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1035/2035] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:11 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:11.650079 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a6ea4-cdec-4400-963a-db30f0cafc75 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:11.670459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a6ea4-cdec-4400-963a-db30f0cafc75 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:11.679531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804baf5c-7484-4d89-967a-4fa548b387a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:11.680605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804baf5c-7484-4d89-967a-4fa548b387a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.680999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804baf5c-7484-4d89-967a-4fa548b387a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.681356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804baf5c-7484-4d89-967a-4fa548b387a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.682557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804baf5c-7484-4d89-967a-4fa548b387a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:11.685480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804baf5c-7484-4d89-967a-4fa548b387a9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:11.702494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1001/2036] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:11.707183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586156a6-0d99-4dfa-8275-9a1a2121923c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:11.720119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586156a6-0d99-4dfa-8275-9a1a2121923c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:11.728713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4f86a-c7a0-4027-8cbe-d6d0369096be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:11.730151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4f86a-c7a0-4027-8cbe-d6d0369096be admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.730658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4f86a-c7a0-4027-8cbe-d6d0369096be admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.731276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4f86a-c7a0-4027-8cbe-d6d0369096be admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.733507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4f86a-c7a0-4027-8cbe-d6d0369096be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:11.735900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4f86a-c7a0-4027-8cbe-d6d0369096be admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:11.756297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1036/2037] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:11.758228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804baf5c-7484-4d89-967a-4fa548b387a9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:11.772936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804baf5c-7484-4d89-967a-4fa548b387a9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:11.780389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ff22b-55e8-4bcc-8ff1-3834d930a2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:11.781936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ff22b-55e8-4bcc-8ff1-3834d930a2d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.782351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ff22b-55e8-4bcc-8ff1-3834d930a2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.782811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ff22b-55e8-4bcc-8ff1-3834d930a2d8 admin admin] PATH_INFO: `/v3/projects/4f8456faac6442a4aeae50d2f49df1bf/users/bb969c40e08b4deda6ea02828130ad58/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.807356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ff22b-55e8-4bcc-8ff1-3834d930a2d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f8456faac6442a4aeae50d2f49df1bf, user_id=bb969c40e08b4deda6ea02828130ad58, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:11.807356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ff22b-55e8-4bcc-8ff1-3834d930a2d8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:11.826913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb11f8-bee0-455c-b0f4-efe2d1a6fac5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.827216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1002/2038] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:11 2022] PUT /identity/v3/projects/4f8456faac6442a4aeae50d2f49df1bf/users/bb969c40e08b4deda6ea02828130ad58/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:11.827556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb11f8-bee0-455c-b0f4-efe2d1a6fac5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.827556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb11f8-bee0-455c-b0f4-efe2d1a6fac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.842651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444a451-6981-4dd7-b323-fc8af6cfc554 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:11.842885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444a451-6981-4dd7-b323-fc8af6cfc554 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:11.843169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444a451-6981-4dd7-b323-fc8af6cfc554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:11.859320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-45fb11f8-bee0-455c-b0f4-efe2d1a6fac5 None None] MFA Rules not processed for user `66ded9372b644c57a9f13fc88ff7fdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:11.861491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-45fb11f8-bee0-455c-b0f4-efe2d1a6fac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:11.888181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7444a451-6981-4dd7-b323-fc8af6cfc554 None None] MFA Rules not processed for user `bb969c40e08b4deda6ea02828130ad58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:11.889508 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7444a451-6981-4dd7-b323-fc8af6cfc554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:11.972632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:11.998360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1037/2039] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:12.012261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1003/2040] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:12.019039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb11f8-bee0-455c-b0f4-efe2d1a6fac5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:12.156993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb11f8-bee0-455c-b0f4-efe2d1a6fac5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:12.166331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866eb02-6523-4614-b1c2-7002fa544c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:12.167408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866eb02-6523-4614-b1c2-7002fa544c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:12.167947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866eb02-6523-4614-b1c2-7002fa544c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:12.168465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866eb02-6523-4614-b1c2-7002fa544c60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:12.169621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866eb02-6523-4614-b1c2-7002fa544c60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:12.172181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866eb02-6523-4614-b1c2-7002fa544c60 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:12.222629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1038/2041] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:12 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:12.223869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444a451-6981-4dd7-b323-fc8af6cfc554 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:12.243630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444a451-6981-4dd7-b323-fc8af6cfc554 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:12.367566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadfe22-8750-43a4-930c-2f46fda52920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:12.368988 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadfe22-8750-43a4-930c-2f46fda52920 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:12.369487 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadfe22-8750-43a4-930c-2f46fda52920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:12.370126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadfe22-8750-43a4-930c-2f46fda52920 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:12.371579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadfe22-8750-43a4-930c-2f46fda52920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:12.377566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadfe22-8750-43a4-930c-2f46fda52920 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:12.394391 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:12.420035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1004/2042] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:12 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:12.426375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866eb02-6523-4614-b1c2-7002fa544c60 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:12.461405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866eb02-6523-4614-b1c2-7002fa544c60 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:12.472263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f53af-416d-4e33-b3d1-06d6455ecd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:12.474378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f53af-416d-4e33-b3d1-06d6455ecd2f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:12.475410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f53af-416d-4e33-b3d1-06d6455ecd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:12.475901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f53af-416d-4e33-b3d1-06d6455ecd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:12.478407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f53af-416d-4e33-b3d1-06d6455ecd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:12.481627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f53af-416d-4e33-b3d1-06d6455ecd2f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:12.504181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1039/2043] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:12.506867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadfe22-8750-43a4-930c-2f46fda52920 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:12.523140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadfe22-8750-43a4-930c-2f46fda52920 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:12.542209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef4254-06fe-48f7-bff7-0eba78c5b609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:12.550895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef4254-06fe-48f7-bff7-0eba78c5b609 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:12.550895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef4254-06fe-48f7-bff7-0eba78c5b609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:12.551834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef4254-06fe-48f7-bff7-0eba78c5b609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:12.556065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef4254-06fe-48f7-bff7-0eba78c5b609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:12.562933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef4254-06fe-48f7-bff7-0eba78c5b609 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:12.592786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1005/2044] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:12.593592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f53af-416d-4e33-b3d1-06d6455ecd2f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:12.606505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f53af-416d-4e33-b3d1-06d6455ecd2f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:12.612111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888d6bb-d1bc-4c34-9de9-09a3eb1a7f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:12.613164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888d6bb-d1bc-4c34-9de9-09a3eb1a7f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:12.613535 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888d6bb-d1bc-4c34-9de9-09a3eb1a7f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:12.613987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888d6bb-d1bc-4c34-9de9-09a3eb1a7f9f admin admin] PATH_INFO: `/v3/projects/961c1bda62fd4fd88825c146146a44bb/users/1109409d58554c2b8b3988c8e371dc4c/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:12.636436 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888d6bb-d1bc-4c34-9de9-09a3eb1a7f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=961c1bda62fd4fd88825c146146a44bb, user_id=1109409d58554c2b8b3988c8e371dc4c, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:12.638901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888d6bb-d1bc-4c34-9de9-09a3eb1a7f9f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:12.663832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1040/2045] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:12 2022] PUT /identity/v3/projects/961c1bda62fd4fd88825c146146a44bb/users/1109409d58554c2b8b3988c8e371dc4c/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:12.676269 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5dd9e-5612-44e3-b255-6bb8cc9e2caf None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:12.676756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5dd9e-5612-44e3-b255-6bb8cc9e2caf None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:12.677247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5dd9e-5612-44e3-b255-6bb8cc9e2caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:12.710686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7ee5dd9e-5612-44e3-b255-6bb8cc9e2caf None None] MFA Rules not processed for user `1109409d58554c2b8b3988c8e371dc4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:12.712280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7ee5dd9e-5612-44e3-b255-6bb8cc9e2caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:12.826065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1006/2046] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:12.832124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888d6bb-d1bc-4c34-9de9-09a3eb1a7f9f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:12.933149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888d6bb-d1bc-4c34-9de9-09a3eb1a7f9f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:12.938699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab4443-14b4-47b6-bd23-90f50534870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:12.939646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab4443-14b4-47b6-bd23-90f50534870a admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:12.940085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab4443-14b4-47b6-bd23-90f50534870a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:12.940567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab4443-14b4-47b6-bd23-90f50534870a admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:12.941440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab4443-14b4-47b6-bd23-90f50534870a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:12.944748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab4443-14b4-47b6-bd23-90f50534870a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:12.992712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1041/2047] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:12 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:12.997218 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5dd9e-5612-44e3-b255-6bb8cc9e2caf None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:13.011093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5dd9e-5612-44e3-b255-6bb8cc9e2caf None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:13.120263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af412f31-9520-4cf5-87fa-fd84ad42a4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:13.121991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af412f31-9520-4cf5-87fa-fd84ad42a4fa admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:13.121991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af412f31-9520-4cf5-87fa-fd84ad42a4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:13.122509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af412f31-9520-4cf5-87fa-fd84ad42a4fa admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:13.123389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af412f31-9520-4cf5-87fa-fd84ad42a4fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:13.126896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af412f31-9520-4cf5-87fa-fd84ad42a4fa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:13.159556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1007/2048] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:12 2022] POST /identity/v3/users => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:24:13.165748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab4443-14b4-47b6-bd23-90f50534870a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:13.186721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab4443-14b4-47b6-bd23-90f50534870a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:13.195556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094cc52-8e1a-4a3d-95e0-242d3a8558e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:13.196605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094cc52-8e1a-4a3d-95e0-242d3a8558e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:13.197061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094cc52-8e1a-4a3d-95e0-242d3a8558e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:13.197520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094cc52-8e1a-4a3d-95e0-242d3a8558e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:13.198799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094cc52-8e1a-4a3d-95e0-242d3a8558e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:13.201011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094cc52-8e1a-4a3d-95e0-242d3a8558e6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:13.221733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1042/2049] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:13.223279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af412f31-9520-4cf5-87fa-fd84ad42a4fa admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:13.238769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af412f31-9520-4cf5-87fa-fd84ad42a4fa admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:13.247264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df77bc8-a540-4c4f-87a5-13d97f1d835e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:13.248688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df77bc8-a540-4c4f-87a5-13d97f1d835e admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:13.249401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df77bc8-a540-4c4f-87a5-13d97f1d835e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:13.249987 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df77bc8-a540-4c4f-87a5-13d97f1d835e admin admin] PATH_INFO: `/v3/projects/4b03d3059d1a46878c947390ae0b6087/users/b0a72ae16ff0435da32fe2d8859d8261/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:13.281408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df77bc8-a540-4c4f-87a5-13d97f1d835e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b03d3059d1a46878c947390ae0b6087, user_id=b0a72ae16ff0435da32fe2d8859d8261, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:13.284904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df77bc8-a540-4c4f-87a5-13d97f1d835e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:13.301444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1008/2050] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:13 2022] PUT /identity/v3/projects/4b03d3059d1a46878c947390ae0b6087/users/b0a72ae16ff0435da32fe2d8859d8261/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:13.306647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094cc52-8e1a-4a3d-95e0-242d3a8558e6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:13.318012 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094cc52-8e1a-4a3d-95e0-242d3a8558e6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:13.353222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:13.423355 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a5991-dbf3-479f-a581-dd4b7310b50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:13.424210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a5991-dbf3-479f-a581-dd4b7310b50f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:13.424593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a5991-dbf3-479f-a581-dd4b7310b50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:13.424983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a5991-dbf3-479f-a581-dd4b7310b50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:13.426297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a5991-dbf3-479f-a581-dd4b7310b50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:13.437746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a5991-dbf3-479f-a581-dd4b7310b50f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:13.452721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1043/2051] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:24:13.458457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df77bc8-a540-4c4f-87a5-13d97f1d835e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:13.474145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df77bc8-a540-4c4f-87a5-13d97f1d835e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:13.483318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7865e-445e-44fe-ac83-5d436aeb3bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:13.484882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7865e-445e-44fe-ac83-5d436aeb3bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:13.485391 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7865e-445e-44fe-ac83-5d436aeb3bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:13.485920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7865e-445e-44fe-ac83-5d436aeb3bf5 admin admin] PATH_INFO: `/v3/projects/4b03d3059d1a46878c947390ae0b6087/users/b0a72ae16ff0435da32fe2d8859d8261/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:13.489005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7865e-445e-44fe-ac83-5d436aeb3bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b03d3059d1a46878c947390ae0b6087, user_id=b0a72ae16ff0435da32fe2d8859d8261, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:13.494666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7865e-445e-44fe-ac83-5d436aeb3bf5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:13.511821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1009/2052] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:13 2022] PUT /identity/v3/projects/4b03d3059d1a46878c947390ae0b6087/users/b0a72ae16ff0435da32fe2d8859d8261/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:13.533103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdf02c-b021-46aa-aee3-26df95de2025 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:13.533584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdf02c-b021-46aa-aee3-26df95de2025 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:13.534122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdf02c-b021-46aa-aee3-26df95de2025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:13.578170 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-79cdf02c-b021-46aa-aee3-26df95de2025 None None] MFA Rules not processed for user `b0a72ae16ff0435da32fe2d8859d8261`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:13.580634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-79cdf02c-b021-46aa-aee3-26df95de2025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:13.747087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1044/2053] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:24:13.767513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7865e-445e-44fe-ac83-5d436aeb3bf5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:13.770940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5aa7865e-445e-44fe-ac83-5d436aeb3bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:13.846122 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:13.927374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7865e-445e-44fe-ac83-5d436aeb3bf5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:13.931848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43246e8b-d3fd-4493-9063-47de82d558f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:13.932678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43246e8b-d3fd-4493-9063-47de82d558f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:13.932972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43246e8b-d3fd-4493-9063-47de82d558f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:13.933421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43246e8b-d3fd-4493-9063-47de82d558f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:14.040133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43246e8b-d3fd-4493-9063-47de82d558f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:14.042825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43246e8b-d3fd-4493-9063-47de82d558f9 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:14.063868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1010/2054] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:24:15.886166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdf02c-b021-46aa-aee3-26df95de2025 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:15.888198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-79cdf02c-b021-46aa-aee3-26df95de2025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:15.912289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdf02c-b021-46aa-aee3-26df95de2025 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:15.917814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef3e8-374f-4bbf-a586-794d9315aafa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:15.918796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef3e8-374f-4bbf-a586-794d9315aafa service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:15.919172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef3e8-374f-4bbf-a586-794d9315aafa service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:15.919749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef3e8-374f-4bbf-a586-794d9315aafa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:16.008282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ef3e8-374f-4bbf-a586-794d9315aafa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:16.011322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ef3e8-374f-4bbf-a586-794d9315aafa service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:16.029477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1045/2055] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:17.035808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43246e8b-d3fd-4493-9063-47de82d558f9 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:17.046178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43246e8b-d3fd-4493-9063-47de82d558f9 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:17.050695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03699ce2-e7c8-454e-b70e-9318fbff8ff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:17.051369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03699ce2-e7c8-454e-b70e-9318fbff8ff3 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:17.051680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03699ce2-e7c8-454e-b70e-9318fbff8ff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:17.051909 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03699ce2-e7c8-454e-b70e-9318fbff8ff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:17.067167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03699ce2-e7c8-454e-b70e-9318fbff8ff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:17.069350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03699ce2-e7c8-454e-b70e-9318fbff8ff3 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:17.080697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1011/2056] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:24:26.021549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=81 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:27.076281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=43 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:27.782387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef3e8-374f-4bbf-a586-794d9315aafa service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:27.990891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef3e8-374f-4bbf-a586-794d9315aafa service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:28.001045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:28.002701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:28.003508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:28.004010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] PATH_INFO: `/v3/users/bb969c40e08b4deda6ea02828130ad58` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:28.006923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb969c40e08b4deda6ea02828130ad58)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:28.010736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:28.092461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb969c40e08b4deda6ea02828130ad58'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:28.103975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb969c40e08b4deda6ea02828130ad58'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:28.105185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb969c40e08b4deda6ea02828130ad58'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:28.116213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb969c40e08b4deda6ea02828130ad58'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:28.133670 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb969c40e08b4deda6ea02828130ad58'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:28.144163 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1046/2057] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:27 2022] DELETE /identity/v3/users/bb969c40e08b4deda6ea02828130ad58 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:28.147534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48703e47-1f08-475c-8554-7f667efffa21 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:28.148087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48703e47-1f08-475c-8554-7f667efffa21 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:28.148212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48703e47-1f08-475c-8554-7f667efffa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:28.170115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-48703e47-1f08-475c-8554-7f667efffa21 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:28.171424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-48703e47-1f08-475c-8554-7f667efffa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:28.259271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1012/2058] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:28.270799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:28.273122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:28.384642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79aef5-6e7a-423f-baba-9d3db49ac5c6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:28.389782 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb82f5b-832c-4e52-bd52-35d946a51be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:28.390427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82f5b-832c-4e52-bd52-35d946a51be9 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:28.390727 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82f5b-832c-4e52-bd52-35d946a51be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:28.391117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82f5b-832c-4e52-bd52-35d946a51be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:28.407833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb82f5b-832c-4e52-bd52-35d946a51be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:28.409859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb82f5b-832c-4e52-bd52-35d946a51be9 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:28.422578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1047/2059] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:24:28.847303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48703e47-1f08-475c-8554-7f667efffa21 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:28.849962 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-48703e47-1f08-475c-8554-7f667efffa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:28.876457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48703e47-1f08-475c-8554-7f667efffa21 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:28.889495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:28.892986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:28.893724 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:28.894606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] PATH_INFO: `/v3/projects/4f8456faac6442a4aeae50d2f49df1bf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:28.898168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f8456faac6442a4aeae50d2f49df1bf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:28.903068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:29.037757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f8456faac6442a4aeae50d2f49df1bf', 'request_id': 'req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.065893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f8456faac6442a4aeae50d2f49df1bf', 'request_id': 'req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.083846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f8456faac6442a4aeae50d2f49df1bf', 'request_id': 'req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.103195 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] The token cache is being invalidate because project 4f8456faac6442a4aeae50d2f49df1bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:29.103758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.105345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.107894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1013/2060] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:28 2022] DELETE /identity/v3/projects/4f8456faac6442a4aeae50d2f49df1bf => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:29.113953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb82f5b-832c-4e52-bd52-35d946a51be9 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:29.127072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb82f5b-832c-4e52-bd52-35d946a51be9 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:29.166975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:29.282766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:29.284448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:29.284904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:29.285403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] PATH_INFO: `/v3/users/1109409d58554c2b8b3988c8e371dc4c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:29.289423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1109409d58554c2b8b3988c8e371dc4c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:29.296833 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:29.412967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1109409d58554c2b8b3988c8e371dc4c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.444646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1109409d58554c2b8b3988c8e371dc4c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.446112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1109409d58554c2b8b3988c8e371dc4c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.464542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1109409d58554c2b8b3988c8e371dc4c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.496087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1109409d58554c2b8b3988c8e371dc4c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:29.514117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1048/2061] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:29 2022] DELETE /identity/v3/users/1109409d58554c2b8b3988c8e371dc4c => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:30.001482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:30.003809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:30.031829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337f07d-1d8b-4577-b7c2-cbd1ae2e4ba1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:30.130761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:30.132955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:30.133397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:30.133892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] PATH_INFO: `/v3/projects/961c1bda62fd4fd88825c146146a44bb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:30.135950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=961c1bda62fd4fd88825c146146a44bb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:30.142740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:30.215467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:30.269753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '961c1bda62fd4fd88825c146146a44bb', 'request_id': 'req-0f476953-cdc3-4656-89c4-da41ddd65f01'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.301814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '961c1bda62fd4fd88825c146146a44bb', 'request_id': 'req-0f476953-cdc3-4656-89c4-da41ddd65f01'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.323391 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '961c1bda62fd4fd88825c146146a44bb', 'request_id': 'req-0f476953-cdc3-4656-89c4-da41ddd65f01'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.356625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] The token cache is being invalidate because project 961c1bda62fd4fd88825c146146a44bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:30.358311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.359640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.362761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1014/2062] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:29 2022] DELETE /identity/v3/projects/961c1bda62fd4fd88825c146146a44bb => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:30.369617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:30.375715 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:30.400865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e66c65-044a-4c9e-9420-7144d8db7145 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:30.567376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:30.569029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:30.569527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:30.569967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] PATH_INFO: `/v3/users/b0a72ae16ff0435da32fe2d8859d8261` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:30.572489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a72ae16ff0435da32fe2d8859d8261)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:30.576475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:30.691828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:30.710684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a72ae16ff0435da32fe2d8859d8261'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.758772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a72ae16ff0435da32fe2d8859d8261'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.760064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a72ae16ff0435da32fe2d8859d8261'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.779641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a72ae16ff0435da32fe2d8859d8261'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.801645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f4843d98-3f69-45e4-b649-1a126b1ca150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a72ae16ff0435da32fe2d8859d8261'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:30.814502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1049/2063] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:30 2022] DELETE /identity/v3/users/b0a72ae16ff0435da32fe2d8859d8261 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:31.397027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:31.398737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:31.427297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f476953-cdc3-4656-89c4-da41ddd65f01 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:31.543908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:31.545326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:31.545658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:31.546189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] PATH_INFO: `/v3/projects/4b03d3059d1a46878c947390ae0b6087` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:31.555776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b03d3059d1a46878c947390ae0b6087)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:31.560366 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:31.675737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b03d3059d1a46878c947390ae0b6087', 'request_id': 'req-5d8a7c09-a715-417c-b2b5-cb6d38495916'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:31.695922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b03d3059d1a46878c947390ae0b6087', 'request_id': 'req-5d8a7c09-a715-417c-b2b5-cb6d38495916'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:31.731287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b03d3059d1a46878c947390ae0b6087', 'request_id': 'req-5d8a7c09-a715-417c-b2b5-cb6d38495916'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:31.767183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] The token cache is being invalidate because project 4b03d3059d1a46878c947390ae0b6087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:31.767892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:31.768993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5d8a7c09-a715-417c-b2b5-cb6d38495916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:31.771808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1015/2064] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:31 2022] DELETE /identity/v3/projects/4b03d3059d1a46878c947390ae0b6087 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:31.781816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3dc7c-f9d6-42e4-b322-0e86a1dc71df None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:31.783182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3dc7c-f9d6-42e4-b322-0e86a1dc71df None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:31.784256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3dc7c-f9d6-42e4-b322-0e86a1dc71df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:31.868433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bfa3dc7c-f9d6-42e4-b322-0e86a1dc71df None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:31.872983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bfa3dc7c-f9d6-42e4-b322-0e86a1dc71df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:32.140458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1050/2065] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:32.158245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3b5b-d293-45cc-ba2e-18340052cc9f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:32.158913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3b5b-d293-45cc-ba2e-18340052cc9f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:32.159661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3b5b-d293-45cc-ba2e-18340052cc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:32.213273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-864f3b5b-d293-45cc-ba2e-18340052cc9f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:32.223499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-864f3b5b-d293-45cc-ba2e-18340052cc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:32.247419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1016/2066] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:24:32.254569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3dc7c-f9d6-42e4-b322-0e86a1dc71df None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:32.269371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3dc7c-f9d6-42e4-b322-0e86a1dc71df None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:32.278480 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19555633-5cc7-4e20-b76a-dcc9b362fa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:32.279791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19555633-5cc7-4e20-b76a-dcc9b362fa66 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:32.280067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19555633-5cc7-4e20-b76a-dcc9b362fa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:32.280507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19555633-5cc7-4e20-b76a-dcc9b362fa66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:32.281290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19555633-5cc7-4e20-b76a-dcc9b362fa66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:24:32.281677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19555633-5cc7-4e20-b76a-dcc9b362fa66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:32.283386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19555633-5cc7-4e20-b76a-dcc9b362fa66 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:32.298740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1051/2067] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:24:32.304657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3b5b-d293-45cc-ba2e-18340052cc9f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:32.317580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3b5b-d293-45cc-ba2e-18340052cc9f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:32.325845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84d283-57c8-4ca1-96da-b953071a68f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:32.327020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84d283-57c8-4ca1-96da-b953071a68f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:32.327540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84d283-57c8-4ca1-96da-b953071a68f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:32.327996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84d283-57c8-4ca1-96da-b953071a68f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:32.329342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84d283-57c8-4ca1-96da-b953071a68f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:32.333531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84d283-57c8-4ca1-96da-b953071a68f5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:32.382496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1017/2068] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:32 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:32.387365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19555633-5cc7-4e20-b76a-dcc9b362fa66 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:32.400592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19555633-5cc7-4e20-b76a-dcc9b362fa66 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:32.541852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd916f1-b489-4918-b01b-5ca567c6a037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:32.543004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd916f1-b489-4918-b01b-5ca567c6a037 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:32.544313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd916f1-b489-4918-b01b-5ca567c6a037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:32.544688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd916f1-b489-4918-b01b-5ca567c6a037 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:32.546213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd916f1-b489-4918-b01b-5ca567c6a037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:32.550480 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd916f1-b489-4918-b01b-5ca567c6a037 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:32.610898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1052/2069] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:32 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:24:32.618480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84d283-57c8-4ca1-96da-b953071a68f5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:32.641682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84d283-57c8-4ca1-96da-b953071a68f5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:32.651381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8def44b-af1b-4c4a-8fb0-bae4d4a017fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:32.652815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8def44b-af1b-4c4a-8fb0-bae4d4a017fa admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:32.653594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8def44b-af1b-4c4a-8fb0-bae4d4a017fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:32.654367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8def44b-af1b-4c4a-8fb0-bae4d4a017fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:32.656265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8def44b-af1b-4c4a-8fb0-bae4d4a017fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:32.659720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8def44b-af1b-4c4a-8fb0-bae4d4a017fa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:32.687185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1018/2070] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:32.693402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd916f1-b489-4918-b01b-5ca567c6a037 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:32.716858 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd916f1-b489-4918-b01b-5ca567c6a037 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:32.730067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6de07-91b1-4b61-b7c5-74beb7098895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:32.731186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6de07-91b1-4b61-b7c5-74beb7098895 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:32.731573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6de07-91b1-4b61-b7c5-74beb7098895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:32.731930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6de07-91b1-4b61-b7c5-74beb7098895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:32.732803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6de07-91b1-4b61-b7c5-74beb7098895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:32.735005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6de07-91b1-4b61-b7c5-74beb7098895 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:32.753756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1053/2071] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:32.757083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8def44b-af1b-4c4a-8fb0-bae4d4a017fa admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:32.765180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:32.772672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8def44b-af1b-4c4a-8fb0-bae4d4a017fa admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:32.780591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03da082-6924-448d-9ba2-051c0948f4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:32.781908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03da082-6924-448d-9ba2-051c0948f4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:32.782373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03da082-6924-448d-9ba2-051c0948f4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:32.782758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03da082-6924-448d-9ba2-051c0948f4c1 admin admin] PATH_INFO: `/v3/projects/87c64a64f6864b84867141833e107e9f/users/6197b2673c3e41eaac7ca07ad56aab1f/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:32.805150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03da082-6924-448d-9ba2-051c0948f4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c64a64f6864b84867141833e107e9f, user_id=6197b2673c3e41eaac7ca07ad56aab1f, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:32.810298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03da082-6924-448d-9ba2-051c0948f4c1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:32.835678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1019/2072] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:32 2022] PUT /identity/v3/projects/87c64a64f6864b84867141833e107e9f/users/6197b2673c3e41eaac7ca07ad56aab1f/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:32.843258 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b5bdc-eebd-4f54-b991-d6a45a4024ab None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:32.844013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b5bdc-eebd-4f54-b991-d6a45a4024ab None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:32.844013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b5bdc-eebd-4f54-b991-d6a45a4024ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:32.873154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1f2b5bdc-eebd-4f54-b991-d6a45a4024ab None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:32.874875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1f2b5bdc-eebd-4f54-b991-d6a45a4024ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:32.887564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:32.997418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1054/2073] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:24:33.009995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03da082-6924-448d-9ba2-051c0948f4c1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:33.012511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d03da082-6924-448d-9ba2-051c0948f4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:33.136308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03da082-6924-448d-9ba2-051c0948f4c1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:33.146490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57898dd6-7109-4390-b9a1-7136f3198c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:33.148160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57898dd6-7109-4390-b9a1-7136f3198c74 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:33.149059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57898dd6-7109-4390-b9a1-7136f3198c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:33.149900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57898dd6-7109-4390-b9a1-7136f3198c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:33.177564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57898dd6-7109-4390-b9a1-7136f3198c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:33.180593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57898dd6-7109-4390-b9a1-7136f3198c74 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:33.203462 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1020/2074] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:34.450553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6c84c-0c8e-48f8-9a77-b031589521af None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:34.451219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6c84c-0c8e-48f8-9a77-b031589521af None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:34.456940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6c84c-0c8e-48f8-9a77-b031589521af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:34.501465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-92d6c84c-0c8e-48f8-9a77-b031589521af None None] MFA Rules not processed for user `6197b2673c3e41eaac7ca07ad56aab1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:34.505781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-92d6c84c-0c8e-48f8-9a77-b031589521af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:34.608422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1055/2075] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:24:34.624725 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57898dd6-7109-4390-b9a1-7136f3198c74 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:34.626244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-57898dd6-7109-4390-b9a1-7136f3198c74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:34.635958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6c84c-0c8e-48f8-9a77-b031589521af None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:34.638202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-92d6c84c-0c8e-48f8-9a77-b031589521af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:34.641294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57898dd6-7109-4390-b9a1-7136f3198c74 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:34.647659 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:34.649091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:34.649091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:34.649091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] PATH_INFO: `/v3/users/1509082e02c74692855549955b3a8ffa` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:34.651803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1509082e02c74692855549955b3a8ffa)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:34.653969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:34.757868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1509082e02c74692855549955b3a8ffa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:34.776336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1509082e02c74692855549955b3a8ffa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:34.777190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1509082e02c74692855549955b3a8ffa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:34.800680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1509082e02c74692855549955b3a8ffa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:34.801663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15354176-4eab-42fd-87ad-7760465128b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1509082e02c74692855549955b3a8ffa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:34.814369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6c84c-0c8e-48f8-9a77-b031589521af None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:34.822251 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:34.823810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:34.824468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:34.825395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:34.829027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:34.839484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1021/2076] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:34 2022] DELETE /identity/v3/users/1509082e02c74692855549955b3a8ffa => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:34.842025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc66c19-60c4-4b0f-a9ac-87c28260c987 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:34.842338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc66c19-60c4-4b0f-a9ac-87c28260c987 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:34.844818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc66c19-60c4-4b0f-a9ac-87c28260c987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:34.862079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:34.867272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:34.876802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-dbc66c19-60c4-4b0f-a9ac-87c28260c987 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:34.879025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dbc66c19-60c4-4b0f-a9ac-87c28260c987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:34.901951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1056/2077] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:24:35.046844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:35.063948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1022/2078] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:35.083572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:35.086073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:35.111984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed0ff6-50e5-4f8f-98bb-fdbced141fea service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:35.213759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:35.234514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380f0dc-9a21-4dc5-83a1-0f80a91c0314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:35.235706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f0dc-9a21-4dc5-83a1-0f80a91c0314 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:35.236289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f0dc-9a21-4dc5-83a1-0f80a91c0314 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:35.236687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f0dc-9a21-4dc5-83a1-0f80a91c0314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:35.251703 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380f0dc-9a21-4dc5-83a1-0f80a91c0314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:35.252156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380f0dc-9a21-4dc5-83a1-0f80a91c0314 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:35.270903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1057/2079] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:24:35.711720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc66c19-60c4-4b0f-a9ac-87c28260c987 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:35.714207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dbc66c19-60c4-4b0f-a9ac-87c28260c987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:35.739149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc66c19-60c4-4b0f-a9ac-87c28260c987 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:35.749301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:35.750851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:35.751325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:35.751818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] PATH_INFO: `/v3/projects/47fc06fdf6f447b3abd99c4d9b7a8183` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:35.756056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] RBAC: Authorizing `identity:delete_project(project_id=47fc06fdf6f447b3abd99c4d9b7a8183)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:35.759996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:35.833151 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47fc06fdf6f447b3abd99c4d9b7a8183', 'request_id': 'req-33fdbf57-4cc9-49ba-9e01-3e783072c87a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:35.844638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47fc06fdf6f447b3abd99c4d9b7a8183', 'request_id': 'req-33fdbf57-4cc9-49ba-9e01-3e783072c87a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:35.852721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47fc06fdf6f447b3abd99c4d9b7a8183', 'request_id': 'req-33fdbf57-4cc9-49ba-9e01-3e783072c87a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:35.868638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] The token cache is being invalidate because project 47fc06fdf6f447b3abd99c4d9b7a8183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:35.868964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:35.870234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33fdbf57-4cc9-49ba-9e01-3e783072c87a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:35.871873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1023/2080] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:35 2022] DELETE /identity/v3/projects/47fc06fdf6f447b3abd99c4d9b7a8183 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:35.877740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5fdd6-e01a-4001-a4b0-06765a033e33 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:35.878464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5fdd6-e01a-4001-a4b0-06765a033e33 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:35.878464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5fdd6-e01a-4001-a4b0-06765a033e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:35.900668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-12e5fdd6-e01a-4001-a4b0-06765a033e33 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:35.902452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-12e5fdd6-e01a-4001-a4b0-06765a033e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:35.995152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1058/2081] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:36.000614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47cf4b-f745-48d7-8cf5-7d719f2c04f8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:36.001133 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47cf4b-f745-48d7-8cf5-7d719f2c04f8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:36.001568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47cf4b-f745-48d7-8cf5-7d719f2c04f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:36.034680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3c47cf4b-f745-48d7-8cf5-7d719f2c04f8 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:36.037060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3c47cf4b-f745-48d7-8cf5-7d719f2c04f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:36.046996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1024/2082] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:36.051781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5fdd6-e01a-4001-a4b0-06765a033e33 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:36.063836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5fdd6-e01a-4001-a4b0-06765a033e33 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:36.070435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd12d22-9377-4ead-91f5-b3f9b5ea91d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:36.071126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd12d22-9377-4ead-91f5-b3f9b5ea91d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:36.071423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd12d22-9377-4ead-91f5-b3f9b5ea91d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:36.071703 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd12d22-9377-4ead-91f5-b3f9b5ea91d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:36.072567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd12d22-9377-4ead-91f5-b3f9b5ea91d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:24:36.072877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd12d22-9377-4ead-91f5-b3f9b5ea91d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:36.074902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd12d22-9377-4ead-91f5-b3f9b5ea91d5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:36.093817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1059/2083] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:24:36.098537 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47cf4b-f745-48d7-8cf5-7d719f2c04f8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:36.109150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47cf4b-f745-48d7-8cf5-7d719f2c04f8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:36.114993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42440dba-acb1-4393-9f3e-13399f7f78ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:36.116026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42440dba-acb1-4393-9f3e-13399f7f78ca admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:36.116513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42440dba-acb1-4393-9f3e-13399f7f78ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:36.116919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42440dba-acb1-4393-9f3e-13399f7f78ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:36.118065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42440dba-acb1-4393-9f3e-13399f7f78ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:36.121328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42440dba-acb1-4393-9f3e-13399f7f78ca admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:36.154131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1025/2084] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:36 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:36.158873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd12d22-9377-4ead-91f5-b3f9b5ea91d5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:36.170253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd12d22-9377-4ead-91f5-b3f9b5ea91d5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:36.247080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b5fd5-87ff-481f-a95d-bdc01b613077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:36.247847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b5fd5-87ff-481f-a95d-bdc01b613077 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:36.248169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b5fd5-87ff-481f-a95d-bdc01b613077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:36.248495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b5fd5-87ff-481f-a95d-bdc01b613077 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:36.249278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b5fd5-87ff-481f-a95d-bdc01b613077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:36.251479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b5fd5-87ff-481f-a95d-bdc01b613077 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:36.275967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1060/2085] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:36 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:24:36.281828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42440dba-acb1-4393-9f3e-13399f7f78ca admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:36.293841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42440dba-acb1-4393-9f3e-13399f7f78ca admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:36.299045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f896e-3220-435a-95ef-b68ed28dee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:36.299731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f896e-3220-435a-95ef-b68ed28dee07 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:36.300017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f896e-3220-435a-95ef-b68ed28dee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:36.300295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f896e-3220-435a-95ef-b68ed28dee07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:36.301052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f896e-3220-435a-95ef-b68ed28dee07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:36.302806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f896e-3220-435a-95ef-b68ed28dee07 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:36.314980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1026/2086] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:36.319750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b5fd5-87ff-481f-a95d-bdc01b613077 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:36.331203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b5fd5-87ff-481f-a95d-bdc01b613077 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:36.337579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1d7c8-fec7-43d2-8724-7afbf84b4b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:36.338308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1d7c8-fec7-43d2-8724-7afbf84b4b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:36.338591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1d7c8-fec7-43d2-8724-7afbf84b4b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:36.338875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1d7c8-fec7-43d2-8724-7afbf84b4b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:36.339649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1d7c8-fec7-43d2-8724-7afbf84b4b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:36.341329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1d7c8-fec7-43d2-8724-7afbf84b4b71 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:36.353579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1061/2087] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:24:36.361097 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f896e-3220-435a-95ef-b68ed28dee07 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:36.380957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f896e-3220-435a-95ef-b68ed28dee07 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:36.389237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4523c-0b71-4189-b0c6-5ae330fefa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:36.390811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4523c-0b71-4189-b0c6-5ae330fefa48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:36.391294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4523c-0b71-4189-b0c6-5ae330fefa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:36.391789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4523c-0b71-4189-b0c6-5ae330fefa48 admin admin] PATH_INFO: `/v3/projects/874512ad2d8246e79d82f1134902112f/users/f5142562b1e04575ad357b7d4968cdac/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:36.418845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4523c-0b71-4189-b0c6-5ae330fefa48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=874512ad2d8246e79d82f1134902112f, user_id=f5142562b1e04575ad357b7d4968cdac, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:36.423731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4523c-0b71-4189-b0c6-5ae330fefa48 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:36.453816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1027/2088] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:36 2022] PUT /identity/v3/projects/874512ad2d8246e79d82f1134902112f/users/f5142562b1e04575ad357b7d4968cdac/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:36.468246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507b798-cfb2-405b-a5e6-4e1e5d3665db None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:36.468890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507b798-cfb2-405b-a5e6-4e1e5d3665db None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:36.469362 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507b798-cfb2-405b-a5e6-4e1e5d3665db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:36.521318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7507b798-cfb2-405b-a5e6-4e1e5d3665db None None] MFA Rules not processed for user `f5142562b1e04575ad357b7d4968cdac`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:36.522865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7507b798-cfb2-405b-a5e6-4e1e5d3665db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:36.616901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:36.664280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1062/2089] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:36.684919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4523c-0b71-4189-b0c6-5ae330fefa48 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:36.687121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1ac4523c-0b71-4189-b0c6-5ae330fefa48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:36.777559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4523c-0b71-4189-b0c6-5ae330fefa48 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:36.781615 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb32fd-f842-41bf-a6c5-5b524fd3d19a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:36.782338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb32fd-f842-41bf-a6c5-5b524fd3d19a service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:36.782622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb32fd-f842-41bf-a6c5-5b524fd3d19a service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:36.782899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb32fd-f842-41bf-a6c5-5b524fd3d19a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:36.793090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb32fd-f842-41bf-a6c5-5b524fd3d19a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:36.794598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb32fd-f842-41bf-a6c5-5b524fd3d19a service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:36.805126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1028/2090] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:38.784441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557801e-7d8a-4ee4-9586-d1ab937481cf None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:38.785004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557801e-7d8a-4ee4-9586-d1ab937481cf None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:38.785004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557801e-7d8a-4ee4-9586-d1ab937481cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:38.809343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2557801e-7d8a-4ee4-9586-d1ab937481cf None None] MFA Rules not processed for user `6197b2673c3e41eaac7ca07ad56aab1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:38.810613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2557801e-7d8a-4ee4-9586-d1ab937481cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:38.886166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1063/2091] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:24:42.869023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb32fd-f842-41bf-a6c5-5b524fd3d19a service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:42.871895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ddbb32fd-f842-41bf-a6c5-5b524fd3d19a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:42.915861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:42.996272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb32fd-f842-41bf-a6c5-5b524fd3d19a service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:43.003058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:43.003959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:43.004242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:43.004519 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] PATH_INFO: `/v3/users/041bdd97691647e893e56ae2184c0271` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:43.006403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=041bdd97691647e893e56ae2184c0271)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:43.008406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:43.152024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041bdd97691647e893e56ae2184c0271'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:43.168554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041bdd97691647e893e56ae2184c0271'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:43.170533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041bdd97691647e893e56ae2184c0271'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:43.198787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041bdd97691647e893e56ae2184c0271'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:43.204371 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041bdd97691647e893e56ae2184c0271'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:43.240488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1029/2092] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:42 2022] DELETE /identity/v3/users/041bdd97691647e893e56ae2184c0271 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:43.242915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8171cdb-db61-4ee8-a7ac-ed0e7325d2f5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:43.243481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8171cdb-db61-4ee8-a7ac-ed0e7325d2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:43.243924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8171cdb-db61-4ee8-a7ac-ed0e7325d2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:43.303569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c8171cdb-db61-4ee8-a7ac-ed0e7325d2f5 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:43.305681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c8171cdb-db61-4ee8-a7ac-ed0e7325d2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:43.439421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1064/2093] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:43.454574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:43.456757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:43.602086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cf036-47dd-4564-8532-5cfcffbc7bca admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:43.607083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974dcd13-4551-4295-ad57-ef5e364cd048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:43.607762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dcd13-4551-4295-ad57-ef5e364cd048 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:43.608047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dcd13-4551-4295-ad57-ef5e364cd048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:43.608332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dcd13-4551-4295-ad57-ef5e364cd048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:43.620923 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974dcd13-4551-4295-ad57-ef5e364cd048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:43.622512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974dcd13-4551-4295-ad57-ef5e364cd048 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:43.636413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1030/2094] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:44.456884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8171cdb-db61-4ee8-a7ac-ed0e7325d2f5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:44.459256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c8171cdb-db61-4ee8-a7ac-ed0e7325d2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:44.505732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8171cdb-db61-4ee8-a7ac-ed0e7325d2f5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:44.515270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:44.519801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:44.521101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:44.522037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] PATH_INFO: `/v3/projects/5963ca6e05ba42c4993767cf1c269438` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:44.525052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5963ca6e05ba42c4993767cf1c269438)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:44.532504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:44.671903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5963ca6e05ba42c4993767cf1c269438', 'request_id': 'req-8e8e3961-3b3a-4909-a517-7739d0266ef4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:44.691096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5963ca6e05ba42c4993767cf1c269438', 'request_id': 'req-8e8e3961-3b3a-4909-a517-7739d0266ef4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:44.722117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5963ca6e05ba42c4993767cf1c269438', 'request_id': 'req-8e8e3961-3b3a-4909-a517-7739d0266ef4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:44.751700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] The token cache is being invalidate because project 5963ca6e05ba42c4993767cf1c269438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:44.752730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:44.754284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:44.763774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974dcd13-4551-4295-ad57-ef5e364cd048 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:44.769861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1065/2095] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:44 2022] DELETE /identity/v3/projects/5963ca6e05ba42c4993767cf1c269438 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:44.775994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974dcd13-4551-4295-ad57-ef5e364cd048 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:44.920289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:44.978832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:44.981081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:44.981929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:44.982392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] PATH_INFO: `/v3/users/ca5dd913f12740098c7148c4aadd77c5` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:44.984780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca5dd913f12740098c7148c4aadd77c5)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:44.989996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:45.139250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca5dd913f12740098c7148c4aadd77c5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:45.156699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca5dd913f12740098c7148c4aadd77c5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:45.157890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca5dd913f12740098c7148c4aadd77c5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:45.190298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca5dd913f12740098c7148c4aadd77c5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:45.191528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b0af1c4-7ab7-4c7c-8e85-4c39ffaaea88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca5dd913f12740098c7148c4aadd77c5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:45.236068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1031/2096] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:44 2022] DELETE /identity/v3/users/ca5dd913f12740098c7148c4aadd77c5 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:45.795766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:45.798193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:45.818083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e3961-3b3a-4909-a517-7739d0266ef4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:45.833587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:45.947579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:45.948614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:45.948952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:45.949231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] PATH_INFO: `/v3/projects/ff9b740bf98a4f72af048e0105f603e2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:45.951570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9b740bf98a4f72af048e0105f603e2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:45.953793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:46.046200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9b740bf98a4f72af048e0105f603e2', 'request_id': 'req-5b8edbf3-62f7-4488-bbb6-ad6c03365880'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:46.066440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9b740bf98a4f72af048e0105f603e2', 'request_id': 'req-5b8edbf3-62f7-4488-bbb6-ad6c03365880'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:46.079853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9b740bf98a4f72af048e0105f603e2', 'request_id': 'req-5b8edbf3-62f7-4488-bbb6-ad6c03365880'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:46.101660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] The token cache is being invalidate because project ff9b740bf98a4f72af048e0105f603e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:46.102231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:46.103341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b8edbf3-62f7-4488-bbb6-ad6c03365880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:46.106351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1066/2097] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:45 2022] DELETE /identity/v3/projects/ff9b740bf98a4f72af048e0105f603e2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:46.114600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ea806-de1a-4d77-adb8-52b834a2dc64 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:46.115944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ea806-de1a-4d77-adb8-52b834a2dc64 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:46.116321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ea806-de1a-4d77-adb8-52b834a2dc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:46.150214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-641ea806-de1a-4d77-adb8-52b834a2dc64 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:46.154188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-641ea806-de1a-4d77-adb8-52b834a2dc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:46.256590 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1032/2098] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:46.277030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf204b-3e1e-4e7e-8fd2-be9eea99d1db None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:46.277546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf204b-3e1e-4e7e-8fd2-be9eea99d1db None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:46.278172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf204b-3e1e-4e7e-8fd2-be9eea99d1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:46.353172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5dcf204b-3e1e-4e7e-8fd2-be9eea99d1db None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:46.363574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5dcf204b-3e1e-4e7e-8fd2-be9eea99d1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:46.415077 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1067/2099] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:46.416976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ea806-de1a-4d77-adb8-52b834a2dc64 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:46.471721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ea806-de1a-4d77-adb8-52b834a2dc64 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:46.496494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae1387-bb69-4b0c-bc91-52b7c13b9256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:46.499899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae1387-bb69-4b0c-bc91-52b7c13b9256 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:46.502182 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae1387-bb69-4b0c-bc91-52b7c13b9256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:46.503659 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae1387-bb69-4b0c-bc91-52b7c13b9256 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:46.505475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae1387-bb69-4b0c-bc91-52b7c13b9256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:24:46.506316 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae1387-bb69-4b0c-bc91-52b7c13b9256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:46.510381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae1387-bb69-4b0c-bc91-52b7c13b9256 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:46.553372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1033/2100] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:24:46.558533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf204b-3e1e-4e7e-8fd2-be9eea99d1db None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:46.580555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf204b-3e1e-4e7e-8fd2-be9eea99d1db None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:46.589784 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5d796-1ae2-48a0-877d-c54e93edefae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:46.590711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5d796-1ae2-48a0-877d-c54e93edefae admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:46.591097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5d796-1ae2-48a0-877d-c54e93edefae admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:46.591472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5d796-1ae2-48a0-877d-c54e93edefae admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:46.592308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5d796-1ae2-48a0-877d-c54e93edefae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:46.595117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5d796-1ae2-48a0-877d-c54e93edefae admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:46.650353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1068/2101] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:46 2022] POST /identity/v3/projects => generated 357 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:24:46.652235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae1387-bb69-4b0c-bc91-52b7c13b9256 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:46.681838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae1387-bb69-4b0c-bc91-52b7c13b9256 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:46.761422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:46.817219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541fed5-af01-47c5-8568-d6d91c36f53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:46.819080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541fed5-af01-47c5-8568-d6d91c36f53f admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:46.819796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541fed5-af01-47c5-8568-d6d91c36f53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:46.820351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541fed5-af01-47c5-8568-d6d91c36f53f admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:46.821804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541fed5-af01-47c5-8568-d6d91c36f53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:46.826317 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541fed5-af01-47c5-8568-d6d91c36f53f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:46.911558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1034/2102] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:46 2022] POST /identity/v3/users => generated 279 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:24:46.911986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5d796-1ae2-48a0-877d-c54e93edefae admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:46.950817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5d796-1ae2-48a0-877d-c54e93edefae admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:46.962117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8719c4a-ba56-418d-b525-a595f3acc3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:46.965015 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8719c4a-ba56-418d-b525-a595f3acc3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:46.966228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8719c4a-ba56-418d-b525-a595f3acc3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:46.967191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8719c4a-ba56-418d-b525-a595f3acc3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:46.968818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8719c4a-ba56-418d-b525-a595f3acc3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:46.973906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8719c4a-ba56-418d-b525-a595f3acc3b0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:47.006081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1069/2103] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:47.010532 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541fed5-af01-47c5-8568-d6d91c36f53f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:47.059008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541fed5-af01-47c5-8568-d6d91c36f53f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:47.080823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548350f2-92e9-40fc-89c7-8f37a8f99c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:47.081505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548350f2-92e9-40fc-89c7-8f37a8f99c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:47.082038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548350f2-92e9-40fc-89c7-8f37a8f99c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:47.083790 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548350f2-92e9-40fc-89c7-8f37a8f99c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:47.084456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548350f2-92e9-40fc-89c7-8f37a8f99c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:47.086882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548350f2-92e9-40fc-89c7-8f37a8f99c7c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:47.103831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1035/2104] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:24:47.105161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8719c4a-ba56-418d-b525-a595f3acc3b0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:47.123781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8719c4a-ba56-418d-b525-a595f3acc3b0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:47.138463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7ab88-2a93-4509-86d5-ba0b1ea8519b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:47.140432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7ab88-2a93-4509-86d5-ba0b1ea8519b admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:47.141343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7ab88-2a93-4509-86d5-ba0b1ea8519b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:47.142337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7ab88-2a93-4509-86d5-ba0b1ea8519b admin admin] PATH_INFO: `/v3/projects/21294970ac314a888c3be4704da2be64/users/3a74f65a7e17471aaab421916e2dacfb/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:47.228257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7ab88-2a93-4509-86d5-ba0b1ea8519b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21294970ac314a888c3be4704da2be64, user_id=3a74f65a7e17471aaab421916e2dacfb, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:47.233754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7ab88-2a93-4509-86d5-ba0b1ea8519b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:47.274270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1070/2105] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:47 2022] PUT /identity/v3/projects/21294970ac314a888c3be4704da2be64/users/3a74f65a7e17471aaab421916e2dacfb/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:47.285615 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3710cb-738b-413f-b86b-cd54aa9f757a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:47.286647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3710cb-738b-413f-b86b-cd54aa9f757a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:47.286647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3710cb-738b-413f-b86b-cd54aa9f757a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:47.351510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4e3710cb-738b-413f-b86b-cd54aa9f757a None None] MFA Rules not processed for user `3a74f65a7e17471aaab421916e2dacfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:47.353001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4e3710cb-738b-413f-b86b-cd54aa9f757a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:47.474975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7ab88-2a93-4509-86d5-ba0b1ea8519b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:47.477862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1036/2106] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:47 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:47.569499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:47.609370 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7ab88-2a93-4509-86d5-ba0b1ea8519b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:47.619639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d99c2b-3896-43d6-8c00-2d5b2bd32e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:47.620692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d99c2b-3896-43d6-8c00-2d5b2bd32e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:47.621395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d99c2b-3896-43d6-8c00-2d5b2bd32e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:47.621985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d99c2b-3896-43d6-8c00-2d5b2bd32e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:47.623234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d99c2b-3896-43d6-8c00-2d5b2bd32e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:47.627061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d99c2b-3896-43d6-8c00-2d5b2bd32e6a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:47.674828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1071/2107] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:47 2022] POST /identity/v3/projects => generated 357 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:47.680241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3710cb-738b-413f-b86b-cd54aa9f757a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:47.693853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3710cb-738b-413f-b86b-cd54aa9f757a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:47.781048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdba84-0b32-4614-ad70-e8d2735ef807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:47.781854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdba84-0b32-4614-ad70-e8d2735ef807 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:47.782142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdba84-0b32-4614-ad70-e8d2735ef807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:47.783313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdba84-0b32-4614-ad70-e8d2735ef807 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:47.783927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdba84-0b32-4614-ad70-e8d2735ef807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:47.787281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdba84-0b32-4614-ad70-e8d2735ef807 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:47.824414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1037/2108] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:47 2022] POST /identity/v3/users => generated 285 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:47.826404 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d99c2b-3896-43d6-8c00-2d5b2bd32e6a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:47.846509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d99c2b-3896-43d6-8c00-2d5b2bd32e6a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:47.852785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19a8e4-1832-445c-a8a5-2daf2c86aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:47.853591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19a8e4-1832-445c-a8a5-2daf2c86aa6d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:47.854501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19a8e4-1832-445c-a8a5-2daf2c86aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:47.855064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19a8e4-1832-445c-a8a5-2daf2c86aa6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:47.855931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19a8e4-1832-445c-a8a5-2daf2c86aa6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:47.858282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19a8e4-1832-445c-a8a5-2daf2c86aa6d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:47.877894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1072/2109] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:47.880064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdba84-0b32-4614-ad70-e8d2735ef807 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:47.899506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdba84-0b32-4614-ad70-e8d2735ef807 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:47.911258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cd0b6-7e44-4835-995c-ee68c73afc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:47.913818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cd0b6-7e44-4835-995c-ee68c73afc46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:47.919860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cd0b6-7e44-4835-995c-ee68c73afc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:47.920854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cd0b6-7e44-4835-995c-ee68c73afc46 admin admin] PATH_INFO: `/v3/projects/6d0217fb43b044a5a564fef107dda738/users/32d05da533de4e948fb321d134e5fc03/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:47.972869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cd0b6-7e44-4835-995c-ee68c73afc46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0217fb43b044a5a564fef107dda738, user_id=32d05da533de4e948fb321d134e5fc03, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:47.977880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cd0b6-7e44-4835-995c-ee68c73afc46 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:48.027452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1038/2110] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:47 2022] PUT /identity/v3/projects/6d0217fb43b044a5a564fef107dda738/users/32d05da533de4e948fb321d134e5fc03/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:48.032637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19a8e4-1832-445c-a8a5-2daf2c86aa6d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:48.059467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19a8e4-1832-445c-a8a5-2daf2c86aa6d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:48.154880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dfcd9-599c-41d8-b79d-14c7fb1d18ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:48.155754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dfcd9-599c-41d8-b79d-14c7fb1d18ec admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:48.156112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dfcd9-599c-41d8-b79d-14c7fb1d18ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:48.156464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dfcd9-599c-41d8-b79d-14c7fb1d18ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:48.157285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590dfcd9-599c-41d8-b79d-14c7fb1d18ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:48.159099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590dfcd9-599c-41d8-b79d-14c7fb1d18ec admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:48.178045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1073/2111] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:48.184252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cd0b6-7e44-4835-995c-ee68c73afc46 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:48.209261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cd0b6-7e44-4835-995c-ee68c73afc46 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:48.220641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3141c-8ac8-4646-a55d-78cc2922725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:48.225473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3141c-8ac8-4646-a55d-78cc2922725c admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:48.226020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3141c-8ac8-4646-a55d-78cc2922725c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:48.226525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3141c-8ac8-4646-a55d-78cc2922725c admin admin] PATH_INFO: `/v3/projects/6d0217fb43b044a5a564fef107dda738/users/32d05da533de4e948fb321d134e5fc03/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:48.231613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3141c-8ac8-4646-a55d-78cc2922725c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0217fb43b044a5a564fef107dda738, user_id=32d05da533de4e948fb321d134e5fc03, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:48.237637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3141c-8ac8-4646-a55d-78cc2922725c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:48.261004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1039/2112] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:48 2022] PUT /identity/v3/projects/6d0217fb43b044a5a564fef107dda738/users/32d05da533de4e948fb321d134e5fc03/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:24:48.286432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54090cee-efba-4c63-8a53-c3d2086675e3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:48.287592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54090cee-efba-4c63-8a53-c3d2086675e3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:48.288344 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54090cee-efba-4c63-8a53-c3d2086675e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:48.339016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-54090cee-efba-4c63-8a53-c3d2086675e3 None None] MFA Rules not processed for user `32d05da533de4e948fb321d134e5fc03`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:48.340468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-54090cee-efba-4c63-8a53-c3d2086675e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:48.468307 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1074/2113] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:48 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:48.483888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3141c-8ac8-4646-a55d-78cc2922725c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:48.485675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6ef3141c-8ac8-4646-a55d-78cc2922725c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:48.498577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:48.616604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3141c-8ac8-4646-a55d-78cc2922725c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:48.621216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2046ad-c2cf-492a-aaea-e0834db53a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:48.622213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2046ad-c2cf-492a-aaea-e0834db53a0b service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:48.622658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2046ad-c2cf-492a-aaea-e0834db53a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:48.623006 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2046ad-c2cf-492a-aaea-e0834db53a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:48.641568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2046ad-c2cf-492a-aaea-e0834db53a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:48.644899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2046ad-c2cf-492a-aaea-e0834db53a0b service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:48.679495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1040/2114] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:48 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:48.899504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54090cee-efba-4c63-8a53-c3d2086675e3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:48.911021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54090cee-efba-4c63-8a53-c3d2086675e3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:48.916827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e9783-f660-420c-83dc-e233c6853fb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:48.917507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e9783-f660-420c-83dc-e233c6853fb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:48.917865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e9783-f660-420c-83dc-e233c6853fb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:48.918143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e9783-f660-420c-83dc-e233c6853fb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:48.990005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:49.044837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e9783-f660-420c-83dc-e233c6853fb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:49.048021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e9783-f660-420c-83dc-e233c6853fb0 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:49.060087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1075/2115] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:48 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:24:51.292138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2046ad-c2cf-492a-aaea-e0834db53a0b service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:51.295585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9e2046ad-c2cf-492a-aaea-e0834db53a0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:51.434703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2046ad-c2cf-492a-aaea-e0834db53a0b service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:51.441029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:51.442991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:51.443384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:51.443814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] PATH_INFO: `/v3/users/f5142562b1e04575ad357b7d4968cdac` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:51.446964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5142562b1e04575ad357b7d4968cdac)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:51.449320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:51.568041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5142562b1e04575ad357b7d4968cdac'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:51.585289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5142562b1e04575ad357b7d4968cdac'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:51.587110 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5142562b1e04575ad357b7d4968cdac'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:51.607544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5142562b1e04575ad357b7d4968cdac'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:51.608429 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5142562b1e04575ad357b7d4968cdac'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:51.630193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1041/2116] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:24:51 2022] DELETE /identity/v3/users/f5142562b1e04575ad357b7d4968cdac => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:51.636986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a5bed-4d03-4a23-97f2-8dc8453e61e8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:51.637766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a5bed-4d03-4a23-97f2-8dc8453e61e8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:51.637931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a5bed-4d03-4a23-97f2-8dc8453e61e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:51.666629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d39a5bed-4d03-4a23-97f2-8dc8453e61e8 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:51.668771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d39a5bed-4d03-4a23-97f2-8dc8453e61e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:51.759571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1076/2117] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:51.769153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:51.770867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:51.784019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:51.865740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8688d1f-72a8-472f-bbaa-62857083bc58 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:51.869783 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2413e5-56f2-4ab5-94c5-4b6329a25bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:51.870477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2413e5-56f2-4ab5-94c5-4b6329a25bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:51.870758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2413e5-56f2-4ab5-94c5-4b6329a25bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:51.871065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2413e5-56f2-4ab5-94c5-4b6329a25bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:51.883019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2413e5-56f2-4ab5-94c5-4b6329a25bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:51.884537 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2413e5-56f2-4ab5-94c5-4b6329a25bb4 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:51.899448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1042/2118] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:24:52.555993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a5bed-4d03-4a23-97f2-8dc8453e61e8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:52.558231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d39a5bed-4d03-4a23-97f2-8dc8453e61e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:52.589410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a5bed-4d03-4a23-97f2-8dc8453e61e8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:52.596438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:52.598126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:52.598625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:52.598912 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] PATH_INFO: `/v3/projects/874512ad2d8246e79d82f1134902112f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:52.600270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=874512ad2d8246e79d82f1134902112f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:52.602296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:52.666112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '874512ad2d8246e79d82f1134902112f', 'request_id': 'req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:52.677327 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '874512ad2d8246e79d82f1134902112f', 'request_id': 'req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:52.684913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '874512ad2d8246e79d82f1134902112f', 'request_id': 'req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:52.704261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] The token cache is being invalidate because project 874512ad2d8246e79d82f1134902112f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:24:52.704625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:52.705864 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6f0a9ea5-17ca-4902-bb9b-e16955bc54ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:24:52.708425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1077/2119] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:24:52 2022] DELETE /identity/v3/projects/874512ad2d8246e79d82f1134902112f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:52.717292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e7e56-e54b-4e41-a8da-07445afaed30 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:52.717943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e7e56-e54b-4e41-a8da-07445afaed30 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:52.718365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e7e56-e54b-4e41-a8da-07445afaed30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:52.772775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1d3e7e56-e54b-4e41-a8da-07445afaed30 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:52.775028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1d3e7e56-e54b-4e41-a8da-07445afaed30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:52.881866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1043/2120] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:52.891801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9c317-0c0d-4e77-904a-90151ad2bb62 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:52.892113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9c317-0c0d-4e77-904a-90151ad2bb62 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:52.892501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9c317-0c0d-4e77-904a-90151ad2bb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:52.916607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-65c9c317-0c0d-4e77-904a-90151ad2bb62 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:52.918254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-65c9c317-0c0d-4e77-904a-90151ad2bb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:52.927034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1078/2121] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:52.933026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e7e56-e54b-4e41-a8da-07445afaed30 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:52.947615 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e7e56-e54b-4e41-a8da-07445afaed30 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:52.952617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b45d63-00d3-4e78-9452-13133ef92016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:52.953297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b45d63-00d3-4e78-9452-13133ef92016 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:52.953602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b45d63-00d3-4e78-9452-13133ef92016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:52.953921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b45d63-00d3-4e78-9452-13133ef92016 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:52.955049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b45d63-00d3-4e78-9452-13133ef92016 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:24:52.955362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b45d63-00d3-4e78-9452-13133ef92016 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:52.957022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b45d63-00d3-4e78-9452-13133ef92016 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:52.974094 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1044/2122] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:24:52.978766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9c317-0c0d-4e77-904a-90151ad2bb62 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:52.989645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9c317-0c0d-4e77-904a-90151ad2bb62 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:52.995276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b4885-19aa-42bf-8227-975b97320433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:52.995963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b4885-19aa-42bf-8227-975b97320433 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:52.996241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b4885-19aa-42bf-8227-975b97320433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:52.996511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b4885-19aa-42bf-8227-975b97320433 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:52.997220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b4885-19aa-42bf-8227-975b97320433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:52.999870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b4885-19aa-42bf-8227-975b97320433 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:53.042381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1079/2123] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:24:52 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:53.043815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b45d63-00d3-4e78-9452-13133ef92016 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:53.058171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b45d63-00d3-4e78-9452-13133ef92016 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:53.145561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7c5a2-51aa-4cfd-9247-8ca394514210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:53.146369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7c5a2-51aa-4cfd-9247-8ca394514210 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:53.146718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7c5a2-51aa-4cfd-9247-8ca394514210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:53.147004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7c5a2-51aa-4cfd-9247-8ca394514210 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:53.148009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7c5a2-51aa-4cfd-9247-8ca394514210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:53.150340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7c5a2-51aa-4cfd-9247-8ca394514210 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:53.177820 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1045/2124] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:24:53 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:24:53.182588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b4885-19aa-42bf-8227-975b97320433 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:53.196508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b4885-19aa-42bf-8227-975b97320433 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:53.202381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bff657-b10f-48cf-907f-ab8c960049a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:53.203137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bff657-b10f-48cf-907f-ab8c960049a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:53.203504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bff657-b10f-48cf-907f-ab8c960049a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:53.203837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bff657-b10f-48cf-907f-ab8c960049a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:53.204659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bff657-b10f-48cf-907f-ab8c960049a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:53.206485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bff657-b10f-48cf-907f-ab8c960049a6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:53.220889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1080/2125] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:53.226539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7c5a2-51aa-4cfd-9247-8ca394514210 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:53.247011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7c5a2-51aa-4cfd-9247-8ca394514210 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:53.256433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a36940-65fa-40a7-8ff0-7544930f46c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:53.257678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a36940-65fa-40a7-8ff0-7544930f46c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:53.258308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a36940-65fa-40a7-8ff0-7544930f46c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:53.258774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a36940-65fa-40a7-8ff0-7544930f46c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:53.260223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a36940-65fa-40a7-8ff0-7544930f46c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:53.263526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a36940-65fa-40a7-8ff0-7544930f46c1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:53.290827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1046/2126] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:24:53.295487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bff657-b10f-48cf-907f-ab8c960049a6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:53.316987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bff657-b10f-48cf-907f-ab8c960049a6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:53.329520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbddfc6-cc2a-4d98-83dc-1f247eb39bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:53.331754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbddfc6-cc2a-4d98-83dc-1f247eb39bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:53.332500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbddfc6-cc2a-4d98-83dc-1f247eb39bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:53.333088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbddfc6-cc2a-4d98-83dc-1f247eb39bd0 admin admin] PATH_INFO: `/v3/projects/5ec8cce6b52548508ef6ef3caef6230f/users/0ccc40cc6df04b32a5163d8265c4ee73/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:53.356704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:53.364626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbddfc6-cc2a-4d98-83dc-1f247eb39bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec8cce6b52548508ef6ef3caef6230f, user_id=0ccc40cc6df04b32a5163d8265c4ee73, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:53.368486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbddfc6-cc2a-4d98-83dc-1f247eb39bd0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:53.391661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1081/2127] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:24:53 2022] PUT /identity/v3/projects/5ec8cce6b52548508ef6ef3caef6230f/users/0ccc40cc6df04b32a5163d8265c4ee73/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:24:53.410950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eb0e4-1638-4acb-aacd-70b22885318d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:53.411463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eb0e4-1638-4acb-aacd-70b22885318d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:53.411870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eb0e4-1638-4acb-aacd-70b22885318d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:53.464009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-367eb0e4-1638-4acb-aacd-70b22885318d None None] MFA Rules not processed for user `0ccc40cc6df04b32a5163d8265c4ee73`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:24:53.466540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-367eb0e4-1638-4acb-aacd-70b22885318d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:53.475985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:24:53.666177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1047/2128] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:24:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:24:53.685461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbddfc6-cc2a-4d98-83dc-1f247eb39bd0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:24:53.687569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-efbddfc6-cc2a-4d98-83dc-1f247eb39bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:24:53.779224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbddfc6-cc2a-4d98-83dc-1f247eb39bd0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:24:53.783839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f93c0-cff8-4ef9-ad1d-6a5fea23a424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:24:53.784929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f93c0-cff8-4ef9-ad1d-6a5fea23a424 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:24:53.785208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f93c0-cff8-4ef9-ad1d-6a5fea23a424 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:24:53.786047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f93c0-cff8-4ef9-ad1d-6a5fea23a424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:24:53.799183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f93c0-cff8-4ef9-ad1d-6a5fea23a424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:24:53.800707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f93c0-cff8-4ef9-ad1d-6a5fea23a424 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:24:53.812282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1082/2129] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:24:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:25:03.661602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=24 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:03.807348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:05.440694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da7df7-812a-4b80-86b5-8a774fe6b9d8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:05.440694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da7df7-812a-4b80-86b5-8a774fe6b9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:05.440694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da7df7-812a-4b80-86b5-8a774fe6b9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:05.545867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-54da7df7-812a-4b80-86b5-8a774fe6b9d8 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:05.548304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-54da7df7-812a-4b80-86b5-8a774fe6b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:05.718084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1048/2130] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:05.811643 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f93c0-cff8-4ef9-ad1d-6a5fea23a424 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:05.815363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-321f93c0-cff8-4ef9-ad1d-6a5fea23a424 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:06.013487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f93c0-cff8-4ef9-ad1d-6a5fea23a424 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:06.025130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b303e-768f-472f-97cf-98508d1b341f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:06.026369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b303e-768f-472f-97cf-98508d1b341f service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:06.026915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b303e-768f-472f-97cf-98508d1b341f service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:06.027519 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b303e-768f-472f-97cf-98508d1b341f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:06.046956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b303e-768f-472f-97cf-98508d1b341f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:06.050505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b303e-768f-472f-97cf-98508d1b341f service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:06.071506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1083/2131] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:25:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 14:25:08.327495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9b209-26cc-419c-8cdb-65359db26e5b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:08.328139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9b209-26cc-419c-8cdb-65359db26e5b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:08.328385 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9b209-26cc-419c-8cdb-65359db26e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:08.359007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8df9b209-26cc-419c-8cdb-65359db26e5b None None] MFA Rules not processed for user `53505ac8ab6142a6bf0c3d6fe1cc721f`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:08.363688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8df9b209-26cc-419c-8cdb-65359db26e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:08.529448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1049/2132] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:10.333170 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73187d35-0221-45f8-8550-03151d3a42ef None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:10.334360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73187d35-0221-45f8-8550-03151d3a42ef None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:10.334807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73187d35-0221-45f8-8550-03151d3a42ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:10.420845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-73187d35-0221-45f8-8550-03151d3a42ef None None] MFA Rules not processed for user `6197b2673c3e41eaac7ca07ad56aab1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:10.424014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-73187d35-0221-45f8-8550-03151d3a42ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:10.666449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1084/2133] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:11.411500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9b209-26cc-419c-8cdb-65359db26e5b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:11.417687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8df9b209-26cc-419c-8cdb-65359db26e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:11.626526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9b209-26cc-419c-8cdb-65359db26e5b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:11.639690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:11.641072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:11.641594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:11.642197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] PATH_INFO: `/v3/users/3a74f65a7e17471aaab421916e2dacfb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:11.649663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a74f65a7e17471aaab421916e2dacfb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:11.654435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:11.661248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:11.774385 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a74f65a7e17471aaab421916e2dacfb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:11.787224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a74f65a7e17471aaab421916e2dacfb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:11.788743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a74f65a7e17471aaab421916e2dacfb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:11.813257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a74f65a7e17471aaab421916e2dacfb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:11.814237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a74f65a7e17471aaab421916e2dacfb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:11.841375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebda69-7121-4a86-95cf-e7bff61f229a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:11.842343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebda69-7121-4a86-95cf-e7bff61f229a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:11.842766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebda69-7121-4a86-95cf-e7bff61f229a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:11.843846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1050/2134] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:25:11 2022] DELETE /identity/v3/users/3a74f65a7e17471aaab421916e2dacfb => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:11.896885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e2ebda69-7121-4a86-95cf-e7bff61f229a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:11.900594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e2ebda69-7121-4a86-95cf-e7bff61f229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:12.004556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:12.023152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:12.331429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1085/2135] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:12.340885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebda69-7121-4a86-95cf-e7bff61f229a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:12.348035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e2ebda69-7121-4a86-95cf-e7bff61f229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:12.410526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:12.414847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa7532-7d3f-4eb7-a8d3-2ed8e1cb74f5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:12.428557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:12.430616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:12.431632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:12.432481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:12.437747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:12.461306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:12.463517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:12.485329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1051/2136] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:25:12.587140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebda69-7121-4a86-95cf-e7bff61f229a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:12.591748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c708b-ebae-4775-b7e7-9023eecc0d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:12.592537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c708b-ebae-4775-b7e7-9023eecc0d58 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:12.592886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c708b-ebae-4775-b7e7-9023eecc0d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:12.593229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c708b-ebae-4775-b7e7-9023eecc0d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:12.634440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c708b-ebae-4775-b7e7-9023eecc0d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:12.635865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c708b-ebae-4775-b7e7-9023eecc0d58 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:12.670428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1086/2137] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:25:13.515556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:13.517596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:13.546605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaab20-7cbf-48b3-9d05-e70dfe2969f5 service placement] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:13.556956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:13.558951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:13.561126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:13.562352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] PATH_INFO: `/v3/projects/21294970ac314a888c3be4704da2be64` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:13.564097 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=21294970ac314a888c3be4704da2be64)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:13.568738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:13.740484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:13.742518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21294970ac314a888c3be4704da2be64', 'request_id': 'req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:13.763393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21294970ac314a888c3be4704da2be64', 'request_id': 'req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:13.781997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21294970ac314a888c3be4704da2be64', 'request_id': 'req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:13.867242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] The token cache is being invalidate because project 21294970ac314a888c3be4704da2be64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:25:13.867242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:13.870421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:13.882687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1052/2138] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:25:13 2022] DELETE /identity/v3/projects/21294970ac314a888c3be4704da2be64 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:13.888755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c708b-ebae-4775-b7e7-9023eecc0d58 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:13.924586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c708b-ebae-4775-b7e7-9023eecc0d58 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:14.108988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:14.110508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:14.111028 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:14.111509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] PATH_INFO: `/v3/users/32d05da533de4e948fb321d134e5fc03` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:14.113409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32d05da533de4e948fb321d134e5fc03)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:14.116252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:14.191810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32d05da533de4e948fb321d134e5fc03'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:14.201891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32d05da533de4e948fb321d134e5fc03'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:14.202745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32d05da533de4e948fb321d134e5fc03'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:14.216441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32d05da533de4e948fb321d134e5fc03'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:14.259410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e7b3b638-fb12-42ce-944b-ea679c69a164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32d05da533de4e948fb321d134e5fc03'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:14.273620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1087/2139] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:25:13 2022] DELETE /identity/v3/users/32d05da533de4e948fb321d134e5fc03 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:15.023886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:15.026727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:15.075105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ab364-adc1-4a20-ae0d-cc58cd3726ae admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:15.242587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:15.244070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:15.244552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:15.245011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] PATH_INFO: `/v3/projects/6d0217fb43b044a5a564fef107dda738` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:15.246666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0217fb43b044a5a564fef107dda738)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:15.249569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:15.328483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0217fb43b044a5a564fef107dda738', 'request_id': 'req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:15.341765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0217fb43b044a5a564fef107dda738', 'request_id': 'req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:15.360395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0217fb43b044a5a564fef107dda738', 'request_id': 'req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:15.395936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] The token cache is being invalidate because project 6d0217fb43b044a5a564fef107dda738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:25:15.398049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:15.400021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6f5ab099-bd1c-4083-aa13-fdc0bfd8f2ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:15.403777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1053/2140] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:25:15 2022] DELETE /identity/v3/projects/6d0217fb43b044a5a564fef107dda738 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:15.412002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012a8e7-a9d7-47cd-bfac-1da547347667 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:15.412672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012a8e7-a9d7-47cd-bfac-1da547347667 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:15.413062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012a8e7-a9d7-47cd-bfac-1da547347667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:15.442462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d012a8e7-a9d7-47cd-bfac-1da547347667 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:15.444308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d012a8e7-a9d7-47cd-bfac-1da547347667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:15.581351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:15.661903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1088/2141] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:25:15.670605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1140ad-7236-492f-8661-a3293765e43c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:15.671691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1140ad-7236-492f-8661-a3293765e43c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:15.672066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1140ad-7236-492f-8661-a3293765e43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:15.712580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4b1140ad-7236-492f-8661-a3293765e43c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:15.716453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4b1140ad-7236-492f-8661-a3293765e43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:15.740919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1054/2142] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:15.746702 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012a8e7-a9d7-47cd-bfac-1da547347667 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:15.774516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012a8e7-a9d7-47cd-bfac-1da547347667 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:15.813378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7077b94-b58b-4a81-8459-84d10086f08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:15.814399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7077b94-b58b-4a81-8459-84d10086f08b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:15.814886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7077b94-b58b-4a81-8459-84d10086f08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:15.815323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7077b94-b58b-4a81-8459-84d10086f08b admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:15.816841 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7077b94-b58b-4a81-8459-84d10086f08b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:25:15.820515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7077b94-b58b-4a81-8459-84d10086f08b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:15.822657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7077b94-b58b-4a81-8459-84d10086f08b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:15.854151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1089/2143] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:25:15.861346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1140ad-7236-492f-8661-a3293765e43c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:15.876476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1140ad-7236-492f-8661-a3293765e43c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:15.884648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86474eb1-4cc5-4c86-a605-e3572cd60bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:15.885753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86474eb1-4cc5-4c86-a605-e3572cd60bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:15.886199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86474eb1-4cc5-4c86-a605-e3572cd60bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:15.886669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86474eb1-4cc5-4c86-a605-e3572cd60bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:15.887416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86474eb1-4cc5-4c86-a605-e3572cd60bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:15.889397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86474eb1-4cc5-4c86-a605-e3572cd60bf3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:15.927237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1055/2144] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:25:15 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:25:15.929606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7077b94-b58b-4a81-8459-84d10086f08b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:15.948143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7077b94-b58b-4a81-8459-84d10086f08b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:16.085489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86474eb1-4cc5-4c86-a605-e3572cd60bf3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:16.091420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-86474eb1-4cc5-4c86-a605-e3572cd60bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:16.114589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a28df-dbbc-4e25-8d30-1d75160e8e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:16.116139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a28df-dbbc-4e25-8d30-1d75160e8e4b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:16.116633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a28df-dbbc-4e25-8d30-1d75160e8e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:16.117488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a28df-dbbc-4e25-8d30-1d75160e8e4b admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:16.118979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a28df-dbbc-4e25-8d30-1d75160e8e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:16.126210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a28df-dbbc-4e25-8d30-1d75160e8e4b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:16.197906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1090/2145] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:25:15 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:25:16.203625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a28df-dbbc-4e25-8d30-1d75160e8e4b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:16.239637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a28df-dbbc-4e25-8d30-1d75160e8e4b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:16.257056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d288a-3aed-49b7-b5d3-86ee17ecdaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:16.257804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d288a-3aed-49b7-b5d3-86ee17ecdaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:16.260032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d288a-3aed-49b7-b5d3-86ee17ecdaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:16.260475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d288a-3aed-49b7-b5d3-86ee17ecdaa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:16.261138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d288a-3aed-49b7-b5d3-86ee17ecdaa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:16.268855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d288a-3aed-49b7-b5d3-86ee17ecdaa2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:16.271789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:16.293534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1091/2146] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:25:16.299983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d288a-3aed-49b7-b5d3-86ee17ecdaa2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:16.318991 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d288a-3aed-49b7-b5d3-86ee17ecdaa2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:16.330340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacbe35-d120-48c9-9676-5674f3bcc108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:16.330340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacbe35-d120-48c9-9676-5674f3bcc108 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:16.330340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacbe35-d120-48c9-9676-5674f3bcc108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:16.330340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacbe35-d120-48c9-9676-5674f3bcc108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:16.331119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacbe35-d120-48c9-9676-5674f3bcc108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:16.335492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacbe35-d120-48c9-9676-5674f3bcc108 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:16.349787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86474eb1-4cc5-4c86-a605-e3572cd60bf3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:16.362599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1092/2147] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:25:16.362892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:16.362892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:16.362892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:16.364496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] PATH_INFO: `/v3/users/0ccc40cc6df04b32a5163d8265c4ee73` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:16.364620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacbe35-d120-48c9-9676-5674f3bcc108 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:16.368937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ccc40cc6df04b32a5163d8265c4ee73)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:16.372973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:16.387837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacbe35-d120-48c9-9676-5674f3bcc108 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:16.397326 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f7fd5-31f6-4a6a-b9c4-2fb71ef9c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:16.399026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f7fd5-31f6-4a6a-b9c4-2fb71ef9c26d admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:16.400092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f7fd5-31f6-4a6a-b9c4-2fb71ef9c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:16.400560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f7fd5-31f6-4a6a-b9c4-2fb71ef9c26d admin admin] PATH_INFO: `/v3/projects/1ce5320b281c4923912677a1587464ff/users/fbefb39404674cbd83e4d5e084810f37/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:16.440039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f7fd5-31f6-4a6a-b9c4-2fb71ef9c26d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce5320b281c4923912677a1587464ff, user_id=fbefb39404674cbd83e4d5e084810f37, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:16.448269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f7fd5-31f6-4a6a-b9c4-2fb71ef9c26d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:16.517215 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1093/2148] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:25:16 2022] PUT /identity/v3/projects/1ce5320b281c4923912677a1587464ff/users/fbefb39404674cbd83e4d5e084810f37/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:16.536363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e749fbf-33f6-4950-b513-18ce83ab0466 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:16.536619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e749fbf-33f6-4950-b513-18ce83ab0466 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:16.536729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e749fbf-33f6-4950-b513-18ce83ab0466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:16.538025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ccc40cc6df04b32a5163d8265c4ee73'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:16.573480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ccc40cc6df04b32a5163d8265c4ee73'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:16.574630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ccc40cc6df04b32a5163d8265c4ee73'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:16.575342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3e749fbf-33f6-4950-b513-18ce83ab0466 None None] MFA Rules not processed for user `fbefb39404674cbd83e4d5e084810f37`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:16.576889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3e749fbf-33f6-4950-b513-18ce83ab0466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:16.603895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ccc40cc6df04b32a5163d8265c4ee73'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:16.605103 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d6b3c0b1-f9cf-4330-94f8-ae9942d579c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ccc40cc6df04b32a5163d8265c4ee73'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:16.629794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1056/2149] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:25:16 2022] DELETE /identity/v3/users/0ccc40cc6df04b32a5163d8265c4ee73 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:25:16.636136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd6734-89d3-4190-b132-98cd92b21e21 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:16.636823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd6734-89d3-4190-b132-98cd92b21e21 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:16.638460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd6734-89d3-4190-b132-98cd92b21e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:16.711807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-98fd6734-89d3-4190-b132-98cd92b21e21 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:16.713070 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-98fd6734-89d3-4190-b132-98cd92b21e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:16.722788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:16.740774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1094/2150] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:25:16.745381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e749fbf-33f6-4950-b513-18ce83ab0466 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:16.749926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3e749fbf-33f6-4950-b513-18ce83ab0466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:16.890339 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1057/2151] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:16.909080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd6734-89d3-4190-b132-98cd92b21e21 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:16.913889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-98fd6734-89d3-4190-b132-98cd92b21e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:16.939350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e749fbf-33f6-4950-b513-18ce83ab0466 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:16.964409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6325f6-03d9-4079-bf0a-2783b6b9c040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:16.965006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6325f6-03d9-4079-bf0a-2783b6b9c040 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:16.965291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6325f6-03d9-4079-bf0a-2783b6b9c040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:16.965569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6325f6-03d9-4079-bf0a-2783b6b9c040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:16.966340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6325f6-03d9-4079-bf0a-2783b6b9c040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:16.968414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6325f6-03d9-4079-bf0a-2783b6b9c040 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:17.048405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1095/2152] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:25:16 2022] POST /identity/v3/projects => generated 387 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:25:17.048405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6325f6-03d9-4079-bf0a-2783b6b9c040 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:17.068725 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd6734-89d3-4190-b132-98cd92b21e21 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:17.085359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:17.222858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec05d6-e54e-45c2-9646-5c16055c61f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:17.224217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec05d6-e54e-45c2-9646-5c16055c61f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:17.232949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6325f6-03d9-4079-bf0a-2783b6b9c040 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:17.233202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec05d6-e54e-45c2-9646-5c16055c61f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:17.233636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec05d6-e54e-45c2-9646-5c16055c61f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:17.244111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ae822-432a-48fb-bcd6-37e90636f74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:17.245278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ae822-432a-48fb-bcd6-37e90636f74a admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:17.245789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ae822-432a-48fb-bcd6-37e90636f74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:17.246256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ae822-432a-48fb-bcd6-37e90636f74a admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:17.248870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ae822-432a-48fb-bcd6-37e90636f74a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:17.252479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ae822-432a-48fb-bcd6-37e90636f74a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:17.286266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec05d6-e54e-45c2-9646-5c16055c61f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:17.289420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec05d6-e54e-45c2-9646-5c16055c61f9 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:17.299734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1096/2153] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:25:17 2022] POST /identity/v3/users => generated 300 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:25:17.305774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ae822-432a-48fb-bcd6-37e90636f74a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:17.334504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1058/2154] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:25:17.353088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ae822-432a-48fb-bcd6-37e90636f74a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:17.370621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e5f4a-26b3-4272-85a8-98b60d1422e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:17.371320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e5f4a-26b3-4272-85a8-98b60d1422e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:17.371610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e5f4a-26b3-4272-85a8-98b60d1422e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:17.371892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e5f4a-26b3-4272-85a8-98b60d1422e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:17.372669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e5f4a-26b3-4272-85a8-98b60d1422e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:17.374592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e5f4a-26b3-4272-85a8-98b60d1422e5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:17.387363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1097/2155] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:25:17.391544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec05d6-e54e-45c2-9646-5c16055c61f9 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:17.418534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec05d6-e54e-45c2-9646-5c16055c61f9 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:17.428258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ea49c-57f3-40e8-a7c5-f5e4fbb414ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:17.429183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ea49c-57f3-40e8-a7c5-f5e4fbb414ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:17.429477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ea49c-57f3-40e8-a7c5-f5e4fbb414ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:17.429792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ea49c-57f3-40e8-a7c5-f5e4fbb414ee admin admin] PATH_INFO: `/v3/projects/19ce8d9fead94fd3ad2c41898035627a/users/093471b993ad4925bc4132cb6384c01c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:17.450449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ea49c-57f3-40e8-a7c5-f5e4fbb414ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ce8d9fead94fd3ad2c41898035627a, user_id=093471b993ad4925bc4132cb6384c01c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:17.453525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ea49c-57f3-40e8-a7c5-f5e4fbb414ee admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:17.476844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1059/2156] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:25:17 2022] PUT /identity/v3/projects/19ce8d9fead94fd3ad2c41898035627a/users/093471b993ad4925bc4132cb6384c01c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:17.477829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e5f4a-26b3-4272-85a8-98b60d1422e5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:17.493230 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e5f4a-26b3-4272-85a8-98b60d1422e5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:17.590974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9a37c-cc7d-4d4a-9db9-82b22bb18092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:17.591921 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9a37c-cc7d-4d4a-9db9-82b22bb18092 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:17.592309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9a37c-cc7d-4d4a-9db9-82b22bb18092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:17.592692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9a37c-cc7d-4d4a-9db9-82b22bb18092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:17.593566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9a37c-cc7d-4d4a-9db9-82b22bb18092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:17.595736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9a37c-cc7d-4d4a-9db9-82b22bb18092 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:17.600456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:17.611600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1098/2157] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:25:17.616610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ea49c-57f3-40e8-a7c5-f5e4fbb414ee admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:17.642178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ea49c-57f3-40e8-a7c5-f5e4fbb414ee admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:17.652364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5b0f5-fbd0-497b-a046-fa59ef652bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:17.654459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5b0f5-fbd0-497b-a046-fa59ef652bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:17.654753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5b0f5-fbd0-497b-a046-fa59ef652bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:17.655036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5b0f5-fbd0-497b-a046-fa59ef652bb2 admin admin] PATH_INFO: `/v3/projects/19ce8d9fead94fd3ad2c41898035627a/users/093471b993ad4925bc4132cb6384c01c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:17.657375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5b0f5-fbd0-497b-a046-fa59ef652bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ce8d9fead94fd3ad2c41898035627a, user_id=093471b993ad4925bc4132cb6384c01c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:17.661136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5b0f5-fbd0-497b-a046-fa59ef652bb2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:17.692076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1060/2158] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:25:17 2022] PUT /identity/v3/projects/19ce8d9fead94fd3ad2c41898035627a/users/093471b993ad4925bc4132cb6384c01c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:17.693379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52437277-bd86-4928-a052-ae54af5d45a6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:17.693679 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52437277-bd86-4928-a052-ae54af5d45a6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:17.694066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52437277-bd86-4928-a052-ae54af5d45a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:17.753371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-52437277-bd86-4928-a052-ae54af5d45a6 None None] MFA Rules not processed for user `093471b993ad4925bc4132cb6384c01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:17.754878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-52437277-bd86-4928-a052-ae54af5d45a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:17.869154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5b0f5-fbd0-497b-a046-fa59ef652bb2 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:17.871735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7fc5b0f5-fbd0-497b-a046-fa59ef652bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:17.950939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1099/2159] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:25:17.967130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52437277-bd86-4928-a052-ae54af5d45a6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:17.971434 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-52437277-bd86-4928-a052-ae54af5d45a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:18.026576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5b0f5-fbd0-497b-a046-fa59ef652bb2 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:18.036311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:18.039368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:18.042188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:18.042188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] PATH_INFO: `/v3/projects/5ec8cce6b52548508ef6ef3caef6230f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:18.043518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec8cce6b52548508ef6ef3caef6230f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:18.054398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:18.089464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52437277-bd86-4928-a052-ae54af5d45a6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:18.094997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa467c-fede-4e73-b0ff-5ccfc5b17652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:18.095826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa467c-fede-4e73-b0ff-5ccfc5b17652 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:18.096234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa467c-fede-4e73-b0ff-5ccfc5b17652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:18.096614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa467c-fede-4e73-b0ff-5ccfc5b17652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:18.111935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa467c-fede-4e73-b0ff-5ccfc5b17652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:18.114322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa467c-fede-4e73-b0ff-5ccfc5b17652 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:18.125748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1100/2160] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:25:18.211597 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec8cce6b52548508ef6ef3caef6230f', 'request_id': 'req-d9952954-d237-4a60-9b97-2483afd3e1d4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.242446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec8cce6b52548508ef6ef3caef6230f', 'request_id': 'req-d9952954-d237-4a60-9b97-2483afd3e1d4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.262330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec8cce6b52548508ef6ef3caef6230f', 'request_id': 'req-d9952954-d237-4a60-9b97-2483afd3e1d4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.306167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] The token cache is being invalidate because project 5ec8cce6b52548508ef6ef3caef6230f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:25:18.306944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.308199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d9952954-d237-4a60-9b97-2483afd3e1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.313593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1061/2161] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:25:17 2022] DELETE /identity/v3/projects/5ec8cce6b52548508ef6ef3caef6230f => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:18.391713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa467c-fede-4e73-b0ff-5ccfc5b17652 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:18.419417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa467c-fede-4e73-b0ff-5ccfc5b17652 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:18.547157 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:18.548517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:18.548900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:18.549365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] PATH_INFO: `/v3/users/fbefb39404674cbd83e4d5e084810f37` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:18.553521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbefb39404674cbd83e4d5e084810f37)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:18.557649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:18.655077 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:18.760102 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbefb39404674cbd83e4d5e084810f37'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.794901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbefb39404674cbd83e4d5e084810f37'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.796932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbefb39404674cbd83e4d5e084810f37'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.835384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbefb39404674cbd83e4d5e084810f37'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.886816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbefb39404674cbd83e4d5e084810f37'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:18.912508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1101/2162] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:25:18 2022] DELETE /identity/v3/users/fbefb39404674cbd83e4d5e084810f37 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:18.914545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719af12-9ea4-4203-afa8-b8fc1007b3c7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:18.915165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719af12-9ea4-4203-afa8-b8fc1007b3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:18.916057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719af12-9ea4-4203-afa8-b8fc1007b3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:18.934979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:18.943766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0719af12-9ea4-4203-afa8-b8fc1007b3c7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:18.949918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0719af12-9ea4-4203-afa8-b8fc1007b3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:19.085148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1062/2163] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:25:19.101695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:19.103511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:19.219559 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622539a2-4f6f-482f-85fa-6c3da98f2aa9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:19.225525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6ed46-e1e4-4322-889f-5db9d43660bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:19.226837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6ed46-e1e4-4322-889f-5db9d43660bc service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:19.227547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6ed46-e1e4-4322-889f-5db9d43660bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:19.228026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6ed46-e1e4-4322-889f-5db9d43660bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:19.250196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6ed46-e1e4-4322-889f-5db9d43660bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:19.254557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6ed46-e1e4-4322-889f-5db9d43660bc service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:19.272783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1102/2164] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:25:19.875267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719af12-9ea4-4203-afa8-b8fc1007b3c7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:19.876833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0719af12-9ea4-4203-afa8-b8fc1007b3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:19.900609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719af12-9ea4-4203-afa8-b8fc1007b3c7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:19.920240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:19.922414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:19.922906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:19.929308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] PATH_INFO: `/v3/projects/1ce5320b281c4923912677a1587464ff` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:19.932039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce5320b281c4923912677a1587464ff)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:19.935777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:20.118595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce5320b281c4923912677a1587464ff', 'request_id': 'req-314002c8-3e56-4145-95e4-5ad3513849a0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.140977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce5320b281c4923912677a1587464ff', 'request_id': 'req-314002c8-3e56-4145-95e4-5ad3513849a0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.155864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce5320b281c4923912677a1587464ff', 'request_id': 'req-314002c8-3e56-4145-95e4-5ad3513849a0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.184513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] The token cache is being invalidate because project 1ce5320b281c4923912677a1587464ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:25:20.187169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.188680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.193586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1063/2165] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:25:19 2022] DELETE /identity/v3/projects/1ce5320b281c4923912677a1587464ff => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:25:20.198703 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6ed46-e1e4-4322-889f-5db9d43660bc service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:20.212937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6ed46-e1e4-4322-889f-5db9d43660bc service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:20.373366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:20.375000 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:20.375597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:20.376155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] PATH_INFO: `/v3/users/093471b993ad4925bc4132cb6384c01c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:20.379098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] RBAC: Authorizing `identity:delete_user(user_id=093471b993ad4925bc4132cb6384c01c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:20.388198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:20.515305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '093471b993ad4925bc4132cb6384c01c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.520601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:20.558656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '093471b993ad4925bc4132cb6384c01c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.559948 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '093471b993ad4925bc4132cb6384c01c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.580630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '093471b993ad4925bc4132cb6384c01c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.652138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1c2c15b0-cdc0-42cd-b4b0-4bb01fc7abad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '093471b993ad4925bc4132cb6384c01c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:20.673468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1103/2166] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:25:20 2022] DELETE /identity/v3/users/093471b993ad4925bc4132cb6384c01c => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:21.176565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:21.178686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:21.220288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314002c8-3e56-4145-95e4-5ad3513849a0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:21.368052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:21.369078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:21.369552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:21.369940 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] PATH_INFO: `/v3/projects/19ce8d9fead94fd3ad2c41898035627a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:21.371516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ce8d9fead94fd3ad2c41898035627a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:21.373786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:21.435547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:21.508840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ce8d9fead94fd3ad2c41898035627a', 'request_id': 'req-713996b5-1f67-48d8-910b-9ce2659b705f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:21.538389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ce8d9fead94fd3ad2c41898035627a', 'request_id': 'req-713996b5-1f67-48d8-910b-9ce2659b705f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:21.567826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ce8d9fead94fd3ad2c41898035627a', 'request_id': 'req-713996b5-1f67-48d8-910b-9ce2659b705f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:21.588485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] The token cache is being invalidate because project 19ce8d9fead94fd3ad2c41898035627a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:25:21.588485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:21.589142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-713996b5-1f67-48d8-910b-9ce2659b705f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:21.595677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1064/2167] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:25:21 2022] DELETE /identity/v3/projects/19ce8d9fead94fd3ad2c41898035627a => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:21.601623 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6696b-5269-41f8-941d-712ada3dab87 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:21.602234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6696b-5269-41f8-941d-712ada3dab87 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:21.604901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6696b-5269-41f8-941d-712ada3dab87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:21.665219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-94b6696b-5269-41f8-941d-712ada3dab87 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:21.670214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-94b6696b-5269-41f8-941d-712ada3dab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:21.861765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1104/2168] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:21.866489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85bc74-7005-49dc-9d16-626b1a2c3777 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:21.866781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85bc74-7005-49dc-9d16-626b1a2c3777 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:21.867048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85bc74-7005-49dc-9d16-626b1a2c3777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:21.889191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ac85bc74-7005-49dc-9d16-626b1a2c3777 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:21.891306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ac85bc74-7005-49dc-9d16-626b1a2c3777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:21.904798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1065/2169] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:21.905787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6696b-5269-41f8-941d-712ada3dab87 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:21.918922 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6696b-5269-41f8-941d-712ada3dab87 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:21.924085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4678452-2a62-4e30-a3ca-27615242eae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:21.924839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4678452-2a62-4e30-a3ca-27615242eae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:21.925525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4678452-2a62-4e30-a3ca-27615242eae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:21.925525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4678452-2a62-4e30-a3ca-27615242eae4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:21.926644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4678452-2a62-4e30-a3ca-27615242eae4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:25:21.926974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4678452-2a62-4e30-a3ca-27615242eae4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:21.929072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4678452-2a62-4e30-a3ca-27615242eae4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:21.948914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85bc74-7005-49dc-9d16-626b1a2c3777 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:21.953592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1105/2170] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:25:21.960219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85bc74-7005-49dc-9d16-626b1a2c3777 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:21.971375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc6dc1-cbb6-49bf-a70d-fbfa55fd3309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:21.972631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc6dc1-cbb6-49bf-a70d-fbfa55fd3309 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:21.973020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc6dc1-cbb6-49bf-a70d-fbfa55fd3309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:21.973882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc6dc1-cbb6-49bf-a70d-fbfa55fd3309 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:21.979439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc6dc1-cbb6-49bf-a70d-fbfa55fd3309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:21.983527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc6dc1-cbb6-49bf-a70d-fbfa55fd3309 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:22.045545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1066/2171] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:25:21 2022] POST /identity/v3/projects => generated 361 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:25:22.052293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4678452-2a62-4e30-a3ca-27615242eae4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:22.088333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4678452-2a62-4e30-a3ca-27615242eae4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:22.300970 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88762c8d-28bf-433f-974f-48500667b8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:22.302007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88762c8d-28bf-433f-974f-48500667b8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:22.302284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88762c8d-28bf-433f-974f-48500667b8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:22.302780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88762c8d-28bf-433f-974f-48500667b8c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:22.303295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88762c8d-28bf-433f-974f-48500667b8c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:22.305351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88762c8d-28bf-433f-974f-48500667b8c0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:22.337754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1106/2172] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:25:22 2022] POST /identity/v3/users => generated 281 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:25:22.341757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc6dc1-cbb6-49bf-a70d-fbfa55fd3309 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:22.358539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc6dc1-cbb6-49bf-a70d-fbfa55fd3309 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:22.365142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e85f0-eaa1-4e13-8599-0688de45e127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:22.366230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e85f0-eaa1-4e13-8599-0688de45e127 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:22.366801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e85f0-eaa1-4e13-8599-0688de45e127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:22.367296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e85f0-eaa1-4e13-8599-0688de45e127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:22.368260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e85f0-eaa1-4e13-8599-0688de45e127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:22.370990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e85f0-eaa1-4e13-8599-0688de45e127 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:22.387917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1067/2173] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:25:22.394968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88762c8d-28bf-433f-974f-48500667b8c0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:22.412264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88762c8d-28bf-433f-974f-48500667b8c0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:22.426976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c71f2f-caa6-4d1b-a9af-c9d8873b84f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:22.427733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c71f2f-caa6-4d1b-a9af-c9d8873b84f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:22.428072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c71f2f-caa6-4d1b-a9af-c9d8873b84f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:22.428353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c71f2f-caa6-4d1b-a9af-c9d8873b84f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:22.429166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c71f2f-caa6-4d1b-a9af-c9d8873b84f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:22.430938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c71f2f-caa6-4d1b-a9af-c9d8873b84f2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:22.452076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1107/2174] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:25:22.461411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e85f0-eaa1-4e13-8599-0688de45e127 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:22.480477 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e85f0-eaa1-4e13-8599-0688de45e127 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:22.484370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733fba0-2a27-4e56-9398-d67195ef03f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:22.485342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733fba0-2a27-4e56-9398-d67195ef03f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:22.485626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733fba0-2a27-4e56-9398-d67195ef03f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:22.485936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733fba0-2a27-4e56-9398-d67195ef03f3 admin admin] PATH_INFO: `/v3/projects/ba550339f9de479f8abb254fd5c1e546/users/b3060faf0bc0462eaee6707ad3ba73d5/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:22.530706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733fba0-2a27-4e56-9398-d67195ef03f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba550339f9de479f8abb254fd5c1e546, user_id=b3060faf0bc0462eaee6707ad3ba73d5, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:22.533458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733fba0-2a27-4e56-9398-d67195ef03f3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:22.576095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1068/2175] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:25:22 2022] PUT /identity/v3/projects/ba550339f9de479f8abb254fd5c1e546/users/b3060faf0bc0462eaee6707ad3ba73d5/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:22.596829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d48d93-c2f3-4223-8d47-6c77fc94cc86 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:22.596829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d48d93-c2f3-4223-8d47-6c77fc94cc86 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:22.596829 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d48d93-c2f3-4223-8d47-6c77fc94cc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:22.654177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-94d48d93-c2f3-4223-8d47-6c77fc94cc86 None None] MFA Rules not processed for user `b3060faf0bc0462eaee6707ad3ba73d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:22.656400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-94d48d93-c2f3-4223-8d47-6c77fc94cc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:22.699662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:22.867305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1108/2176] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:25:22.967680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733fba0-2a27-4e56-9398-d67195ef03f3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:22.970472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6733fba0-2a27-4e56-9398-d67195ef03f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:23.114430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733fba0-2a27-4e56-9398-d67195ef03f3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:23.121253 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2974aa-b5d9-4b5c-aa42-40ca15019b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:23.122622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2974aa-b5d9-4b5c-aa42-40ca15019b03 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:23.123124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2974aa-b5d9-4b5c-aa42-40ca15019b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:23.123616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2974aa-b5d9-4b5c-aa42-40ca15019b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:23.144543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2974aa-b5d9-4b5c-aa42-40ca15019b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:23.147475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2974aa-b5d9-4b5c-aa42-40ca15019b03 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:23.169877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1069/2177] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:25:23.189592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d48d93-c2f3-4223-8d47-6c77fc94cc86 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:23.353650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d48d93-c2f3-4223-8d47-6c77fc94cc86 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:23.368644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:23.370806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:23.371594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:23.372337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] PATH_INFO: `/v3/users/b3060faf0bc0462eaee6707ad3ba73d5` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:23.375182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3060faf0bc0462eaee6707ad3ba73d5)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:23.379385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:23.520372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3060faf0bc0462eaee6707ad3ba73d5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:23.544843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3060faf0bc0462eaee6707ad3ba73d5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:23.546147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3060faf0bc0462eaee6707ad3ba73d5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:23.571137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3060faf0bc0462eaee6707ad3ba73d5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:23.614826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3060faf0bc0462eaee6707ad3ba73d5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:23.629732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1109/2178] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:25:23 2022] DELETE /identity/v3/users/b3060faf0bc0462eaee6707ad3ba73d5 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:23.636815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44933265-fb7b-4d7c-982d-b4264c51d51d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:23.637683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44933265-fb7b-4d7c-982d-b4264c51d51d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:23.637875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44933265-fb7b-4d7c-982d-b4264c51d51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:23.684001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-44933265-fb7b-4d7c-982d-b4264c51d51d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:23.687524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-44933265-fb7b-4d7c-982d-b4264c51d51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:23.767252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:23.777217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1070/2179] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:23.788780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:23.790423 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:23.865804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:23.886402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe33cf-10f2-493e-8b48-7489c61a26a8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:23.890899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9da79e-09a0-4f0f-a335-fcc57e05a465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:23.891888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9da79e-09a0-4f0f-a335-fcc57e05a465 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:23.892300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9da79e-09a0-4f0f-a335-fcc57e05a465 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:23.892689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9da79e-09a0-4f0f-a335-fcc57e05a465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:23.907637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9da79e-09a0-4f0f-a335-fcc57e05a465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:23.909250 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9da79e-09a0-4f0f-a335-fcc57e05a465 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:23.922871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1110/2180] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:25:25.374558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44933265-fb7b-4d7c-982d-b4264c51d51d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:25.376220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-44933265-fb7b-4d7c-982d-b4264c51d51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:25.392200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44933265-fb7b-4d7c-982d-b4264c51d51d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:25.398663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:25.399850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:25.400330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:25.400806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] PATH_INFO: `/v3/projects/ba550339f9de479f8abb254fd5c1e546` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:25.402729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba550339f9de479f8abb254fd5c1e546)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:25.405229 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:25.494911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba550339f9de479f8abb254fd5c1e546', 'request_id': 'req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:25.512897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba550339f9de479f8abb254fd5c1e546', 'request_id': 'req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:25.522984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba550339f9de479f8abb254fd5c1e546', 'request_id': 'req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:25.568224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] The token cache is being invalidate because project ba550339f9de479f8abb254fd5c1e546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:25:25.568911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:25.570094 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f47568d4-e35d-4403-bc9d-2e579dd9f3f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:25:25.572347 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1071/2181] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:25:25 2022] DELETE /identity/v3/projects/ba550339f9de479f8abb254fd5c1e546 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:25.580524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920679a3-3758-4bf7-b491-400fad48b2c2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:25.581974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920679a3-3758-4bf7-b491-400fad48b2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:25.582626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920679a3-3758-4bf7-b491-400fad48b2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:25.621588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-920679a3-3758-4bf7-b491-400fad48b2c2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:25.624206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-920679a3-3758-4bf7-b491-400fad48b2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:25.762799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1111/2182] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:25.772274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72268e7-2402-44e3-b1ac-9390c5b3d823 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:25.772794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72268e7-2402-44e3-b1ac-9390c5b3d823 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:25.773303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72268e7-2402-44e3-b1ac-9390c5b3d823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:25.811526 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e72268e7-2402-44e3-b1ac-9390c5b3d823 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:25.813895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e72268e7-2402-44e3-b1ac-9390c5b3d823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:25.831778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1072/2183] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:25.836002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920679a3-3758-4bf7-b491-400fad48b2c2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:25.852065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920679a3-3758-4bf7-b491-400fad48b2c2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:25.858303 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edc0cb-33a8-4580-b4a7-62e1f27d19f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:25.858774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edc0cb-33a8-4580-b4a7-62e1f27d19f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:25.859056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edc0cb-33a8-4580-b4a7-62e1f27d19f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:25.859330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edc0cb-33a8-4580-b4a7-62e1f27d19f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:25.860342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edc0cb-33a8-4580-b4a7-62e1f27d19f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:25:25.860661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edc0cb-33a8-4580-b4a7-62e1f27d19f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:25.863108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edc0cb-33a8-4580-b4a7-62e1f27d19f2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:25.887991 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1112/2184] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:25:25.889592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72268e7-2402-44e3-b1ac-9390c5b3d823 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:25.906703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72268e7-2402-44e3-b1ac-9390c5b3d823 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:25.915681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9cbdd-4e6a-4394-9b9f-9c5533dfbbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:25.916761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9cbdd-4e6a-4394-9b9f-9c5533dfbbfb admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:25.917256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9cbdd-4e6a-4394-9b9f-9c5533dfbbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:25.917813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9cbdd-4e6a-4394-9b9f-9c5533dfbbfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:25.919134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9cbdd-4e6a-4394-9b9f-9c5533dfbbfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:25.922953 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9cbdd-4e6a-4394-9b9f-9c5533dfbbfb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:25.986880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1073/2185] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:25:25 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:25:26.000568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edc0cb-33a8-4580-b4a7-62e1f27d19f2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:26.020530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edc0cb-33a8-4580-b4a7-62e1f27d19f2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:26.178704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee988ae8-36e2-4ca4-9a47-cd4efa2602c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:26.179996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee988ae8-36e2-4ca4-9a47-cd4efa2602c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:26.180681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee988ae8-36e2-4ca4-9a47-cd4efa2602c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:26.181179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee988ae8-36e2-4ca4-9a47-cd4efa2602c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:26.182675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee988ae8-36e2-4ca4-9a47-cd4efa2602c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:26.188920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee988ae8-36e2-4ca4-9a47-cd4efa2602c7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:26.236552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1113/2186] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:25:25 2022] POST /identity/v3/users => generated 284 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:25:26.243140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9cbdd-4e6a-4394-9b9f-9c5533dfbbfb admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:26.261596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9cbdd-4e6a-4394-9b9f-9c5533dfbbfb admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:26.272889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b1ccf-f5e1-4ceb-a6cf-12007bfc6526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:26.278577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b1ccf-f5e1-4ceb-a6cf-12007bfc6526 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:26.279008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b1ccf-f5e1-4ceb-a6cf-12007bfc6526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:26.280342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b1ccf-f5e1-4ceb-a6cf-12007bfc6526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:26.280823 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b1ccf-f5e1-4ceb-a6cf-12007bfc6526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:26.284871 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b1ccf-f5e1-4ceb-a6cf-12007bfc6526 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:26.308338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1074/2187] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:25:26.313878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee988ae8-36e2-4ca4-9a47-cd4efa2602c7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:26.333934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee988ae8-36e2-4ca4-9a47-cd4efa2602c7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:26.342253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf138077-9e0d-4798-a043-90bb1ebc1a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:26.342942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf138077-9e0d-4798-a043-90bb1ebc1a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:26.343282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf138077-9e0d-4798-a043-90bb1ebc1a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:26.343892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf138077-9e0d-4798-a043-90bb1ebc1a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:26.345079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf138077-9e0d-4798-a043-90bb1ebc1a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:26.351196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf138077-9e0d-4798-a043-90bb1ebc1a5f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:26.371844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1114/2188] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:25:26.379483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b1ccf-f5e1-4ceb-a6cf-12007bfc6526 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:26.405013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b1ccf-f5e1-4ceb-a6cf-12007bfc6526 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:26.415919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478db8c-7811-4da2-afb5-4c1c0cdfc34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:26.418220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478db8c-7811-4da2-afb5-4c1c0cdfc34b admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:26.419060 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478db8c-7811-4da2-afb5-4c1c0cdfc34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:26.419839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478db8c-7811-4da2-afb5-4c1c0cdfc34b admin admin] PATH_INFO: `/v3/projects/1f731c5517b047d4aca77025cadbb82d/users/33aa8e4e217c486db10be52b7c7bb44b/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:26.458274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478db8c-7811-4da2-afb5-4c1c0cdfc34b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f731c5517b047d4aca77025cadbb82d, user_id=33aa8e4e217c486db10be52b7c7bb44b, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:26.464915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478db8c-7811-4da2-afb5-4c1c0cdfc34b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:26.504015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1075/2189] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:25:26 2022] PUT /identity/v3/projects/1f731c5517b047d4aca77025cadbb82d/users/33aa8e4e217c486db10be52b7c7bb44b/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:25:26.525741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02e4c-634a-46a2-a7ea-fbb3f475358a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:26.526031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02e4c-634a-46a2-a7ea-fbb3f475358a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:26.526397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02e4c-634a-46a2-a7ea-fbb3f475358a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:26.587314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-56c02e4c-634a-46a2-a7ea-fbb3f475358a None None] MFA Rules not processed for user `33aa8e4e217c486db10be52b7c7bb44b`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:26.588601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-56c02e4c-634a-46a2-a7ea-fbb3f475358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:26.666083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:26.699419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1115/2190] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:26.712194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478db8c-7811-4da2-afb5-4c1c0cdfc34b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:26.714512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6478db8c-7811-4da2-afb5-4c1c0cdfc34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:26.858121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478db8c-7811-4da2-afb5-4c1c0cdfc34b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:26.862272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec4f68-8ecf-45f7-8f03-6e834e6ef9b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41a357957bc47739b7bd6f21d159d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:26.863698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec4f68-8ecf-45f7-8f03-6e834e6ef9b1 service glance] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:26.863984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec4f68-8ecf-45f7-8f03-6e834e6ef9b1 service glance] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:26.864257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec4f68-8ecf-45f7-8f03-6e834e6ef9b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:26.874712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec4f68-8ecf-45f7-8f03-6e834e6ef9b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:26.876166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec4f68-8ecf-45f7-8f03-6e834e6ef9b1 service glance] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:26.888109 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1076/2191] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:25:27.661119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b6970-0444-4641-837c-0ca2bdba8818 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:27.661978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b6970-0444-4641-837c-0ca2bdba8818 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:27.661978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b6970-0444-4641-837c-0ca2bdba8818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:27.682159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-359b6970-0444-4641-837c-0ca2bdba8818 None None] MFA Rules not processed for user `86f9bcac527c45d6bc81b70e26544924`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:27.683396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-359b6970-0444-4641-837c-0ca2bdba8818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:27.764407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1116/2192] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:33.025968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e71ae-92c6-4d25-ab82-d103705a3406 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:33.026924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e71ae-92c6-4d25-ab82-d103705a3406 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:33.027512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e71ae-92c6-4d25-ab82-d103705a3406 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:33.029561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1077/2193] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:25:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:25:33.049113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faae040-cb3f-4721-8205-d4f8e25b10ef None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:33.051400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faae040-cb3f-4721-8205-d4f8e25b10ef None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:33.052558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faae040-cb3f-4721-8205-d4f8e25b10ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:33.136901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5faae040-cb3f-4721-8205-d4f8e25b10ef None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:33.138301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5faae040-cb3f-4721-8205-d4f8e25b10ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:33.276554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1117/2194] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:25:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:33.293735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e71ae-92c6-4d25-ab82-d103705a3406 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:33.302970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:33.314535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e71ae-92c6-4d25-ab82-d103705a3406 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:33.320650 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576d453-cd04-4706-a388-3d8de6989d6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41a357957bc47739b7bd6f21d159d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:33.321548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576d453-cd04-4706-a388-3d8de6989d6d service glance] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:33.321932 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576d453-cd04-4706-a388-3d8de6989d6d service glance] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:33.322275 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576d453-cd04-4706-a388-3d8de6989d6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:33.338159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576d453-cd04-4706-a388-3d8de6989d6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:33.339992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576d453-cd04-4706-a388-3d8de6989d6d service glance] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:33.365524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1078/2195] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:25:33.471096 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9a177-1c54-4d19-9e7a-c82a3b0614f0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:33.471353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9a177-1c54-4d19-9e7a-c82a3b0614f0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:33.472023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9a177-1c54-4d19-9e7a-c82a3b0614f0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:33.473857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1118/2196] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:25:33.480984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dc0c8-8a3a-4178-b38a-5251245caba1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:33.481422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dc0c8-8a3a-4178-b38a-5251245caba1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:33.481734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dc0c8-8a3a-4178-b38a-5251245caba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:33.508431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b83dc0c8-8a3a-4178-b38a-5251245caba1 None None] MFA Rules not processed for user `59cd9d922204496d8ea185716b832481`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:33.510031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b83dc0c8-8a3a-4178-b38a-5251245caba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:33.589452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1079/2197] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:25:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:33.605616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9a177-1c54-4d19-9e7a-c82a3b0614f0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:33.607890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a2d9a177-1c54-4d19-9e7a-c82a3b0614f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:33.630589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9a177-1c54-4d19-9e7a-c82a3b0614f0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:33.634155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc133f1-bce4-40f0-ba8c-6a675036f569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:33.634820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc133f1-bce4-40f0-ba8c-6a675036f569 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:33.635101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc133f1-bce4-40f0-ba8c-6a675036f569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:33.635376 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc133f1-bce4-40f0-ba8c-6a675036f569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:33.644828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc133f1-bce4-40f0-ba8c-6a675036f569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:33.646405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc133f1-bce4-40f0-ba8c-6a675036f569 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:33.661287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1119/2198] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:25:33.816588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be753bd5-ae31-4aa8-a736-debbc3d1a43a None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:33.816975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be753bd5-ae31-4aa8-a736-debbc3d1a43a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:33.817413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be753bd5-ae31-4aa8-a736-debbc3d1a43a None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:33.820281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1080/2199] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:25:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:25:33.827411 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7900c95-f581-44b5-a123-f9cdaf835ad0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:33.828213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7900c95-f581-44b5-a123-f9cdaf835ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:33.830377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7900c95-f581-44b5-a123-f9cdaf835ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:33.874007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e7900c95-f581-44b5-a123-f9cdaf835ad0 None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:33.875900 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e7900c95-f581-44b5-a123-f9cdaf835ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:33.892010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1120/2200] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:25:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:25:33.905471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be753bd5-ae31-4aa8-a736-debbc3d1a43a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:33.920934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be753bd5-ae31-4aa8-a736-debbc3d1a43a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:33.925501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759260ae-297e-450e-9d5a-89487119c4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:33.926847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759260ae-297e-450e-9d5a-89487119c4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:33.926847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759260ae-297e-450e-9d5a-89487119c4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:33.926847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759260ae-297e-450e-9d5a-89487119c4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:33.938366 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759260ae-297e-450e-9d5a-89487119c4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:33.939810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759260ae-297e-450e-9d5a-89487119c4c9 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:33.958933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1081/2201] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:25:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:25:43.862686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:43.951730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=34 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:25:48.541873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cdada-c79b-4e75-a1e7-470ab2943a96 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:48.543377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cdada-c79b-4e75-a1e7-470ab2943a96 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:48.544114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cdada-c79b-4e75-a1e7-470ab2943a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:48.582039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-849cdada-c79b-4e75-a1e7-470ab2943a96 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:48.584350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-849cdada-c79b-4e75-a1e7-470ab2943a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:48.713918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1121/2202] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:25:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:48.732990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759260ae-297e-450e-9d5a-89487119c4c9 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:25:48.735336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-759260ae-297e-450e-9d5a-89487119c4c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:48.847752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759260ae-297e-450e-9d5a-89487119c4c9 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:25:48.855622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89124ca9-57e1-4161-a1ed-a188cd688739 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:25:48.856498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89124ca9-57e1-4161-a1ed-a188cd688739 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:48.856961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89124ca9-57e1-4161-a1ed-a188cd688739 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:48.857512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89124ca9-57e1-4161-a1ed-a188cd688739 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:48.868213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89124ca9-57e1-4161-a1ed-a188cd688739 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:25:48.869944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89124ca9-57e1-4161-a1ed-a188cd688739 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:25:48.879838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1082/2203] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:25:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:25:55.521757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf61f0-9876-4346-9815-ceafb53c1583 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:25:55.522770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf61f0-9876-4346-9815-ceafb53c1583 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:25:55.523270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf61f0-9876-4346-9815-ceafb53c1583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:25:55.545519 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b9bf61f0-9876-4346-9815-ceafb53c1583 None None] MFA Rules not processed for user `66ded9372b644c57a9f13fc88ff7fdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:25:55.547477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b9bf61f0-9876-4346-9815-ceafb53c1583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:25:55.629115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1122/2204] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:25:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:25:58.875795 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=32 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:05.620865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=50 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:12.467364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb943885-3ef2-417a-ba9e-ff10c3a700a5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:12.468001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb943885-3ef2-417a-ba9e-ff10c3a700a5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:12.468410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb943885-3ef2-417a-ba9e-ff10c3a700a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:12.495372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fb943885-3ef2-417a-ba9e-ff10c3a700a5 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:12.497289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fb943885-3ef2-417a-ba9e-ff10c3a700a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:12.516580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1083/2205] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:26:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:12.546812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf61f0-9876-4346-9815-ceafb53c1583 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:12.562544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf61f0-9876-4346-9815-ceafb53c1583 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:12.570636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3196385-15bc-4e7f-8a1e-4b7ccfe66e0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:12.571299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196385-15bc-4e7f-8a1e-4b7ccfe66e0e service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:12.571586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196385-15bc-4e7f-8a1e-4b7ccfe66e0e service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:12.571862 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196385-15bc-4e7f-8a1e-4b7ccfe66e0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:12.587711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3196385-15bc-4e7f-8a1e-4b7ccfe66e0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:12.590545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3196385-15bc-4e7f-8a1e-4b7ccfe66e0e service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:12.607402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1123/2206] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:26:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:26:17.042300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355d975-ec2b-4b29-bd3c-a28c21186eac None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:17.042967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355d975-ec2b-4b29-bd3c-a28c21186eac None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:17.043390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355d975-ec2b-4b29-bd3c-a28c21186eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:17.085197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4355d975-ec2b-4b29-bd3c-a28c21186eac None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:17.087040 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4355d975-ec2b-4b29-bd3c-a28c21186eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:17.107945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1084/2207] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 6 14:26:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:17.126115 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3196385-15bc-4e7f-8a1e-4b7ccfe66e0e service swift] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:17.178569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3196385-15bc-4e7f-8a1e-4b7ccfe66e0e service swift] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:17.237398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc006127-2fd1-4a00-bb81-534d47bede73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:17.239248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc006127-2fd1-4a00-bb81-534d47bede73 service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:17.247262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc006127-2fd1-4a00-bb81-534d47bede73 service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:17.248143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc006127-2fd1-4a00-bb81-534d47bede73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:17.274280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc006127-2fd1-4a00-bb81-534d47bede73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:17.279372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc006127-2fd1-4a00-bb81-534d47bede73 service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:17.307143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1124/2208] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:26:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 14:26:18.381639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355d975-ec2b-4b29-bd3c-a28c21186eac None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:18.474889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355d975-ec2b-4b29-bd3c-a28c21186eac None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:18.481212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:18.482686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:18.483252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:18.483741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] PATH_INFO: `/v3/users/33aa8e4e217c486db10be52b7c7bb44b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:18.485450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33aa8e4e217c486db10be52b7c7bb44b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:18.489181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:18.600972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33aa8e4e217c486db10be52b7c7bb44b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:18.612780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33aa8e4e217c486db10be52b7c7bb44b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:18.613907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33aa8e4e217c486db10be52b7c7bb44b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:18.636564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33aa8e4e217c486db10be52b7c7bb44b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:18.637465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33aa8e4e217c486db10be52b7c7bb44b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:18.657917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1085/2209] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:26:18 2022] DELETE /identity/v3/users/33aa8e4e217c486db10be52b7c7bb44b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:26:18.665381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5692d-bdb5-424c-93df-ebcc94b252a2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:18.667182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5692d-bdb5-424c-93df-ebcc94b252a2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:18.668153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5692d-bdb5-424c-93df-ebcc94b252a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:18.706347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-40b5692d-bdb5-424c-93df-ebcc94b252a2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:18.709262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-40b5692d-bdb5-424c-93df-ebcc94b252a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:18.856760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1125/2210] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:18.871666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:18.875315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:19.030956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35b99-cd45-4dff-8235-c5800240e5e5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:19.045037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb5447-359a-49c5-ac45-ea9a7f7cdd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:19.046396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb5447-359a-49c5-ac45-ea9a7f7cdd72 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:19.046998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb5447-359a-49c5-ac45-ea9a7f7cdd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:19.047522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb5447-359a-49c5-ac45-ea9a7f7cdd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:19.068366 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb5447-359a-49c5-ac45-ea9a7f7cdd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:19.071163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb5447-359a-49c5-ac45-ea9a7f7cdd72 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:19.090540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1086/2211] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:26:19.649006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5692d-bdb5-424c-93df-ebcc94b252a2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:19.650903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-40b5692d-bdb5-424c-93df-ebcc94b252a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:19.690881 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5692d-bdb5-424c-93df-ebcc94b252a2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:19.697672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:19.699898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:19.700584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:19.701029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] PATH_INFO: `/v3/projects/1f731c5517b047d4aca77025cadbb82d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:19.702749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f731c5517b047d4aca77025cadbb82d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:19.716654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:19.826075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f731c5517b047d4aca77025cadbb82d', 'request_id': 'req-4014af9d-9058-4c50-a193-1e1d49e2d407'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:19.844441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f731c5517b047d4aca77025cadbb82d', 'request_id': 'req-4014af9d-9058-4c50-a193-1e1d49e2d407'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:19.860108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f731c5517b047d4aca77025cadbb82d', 'request_id': 'req-4014af9d-9058-4c50-a193-1e1d49e2d407'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:19.875792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] The token cache is being invalidate because project 1f731c5517b047d4aca77025cadbb82d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:26:19.876978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:19.878514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4014af9d-9058-4c50-a193-1e1d49e2d407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:19.880726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1126/2212] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:26:19 2022] DELETE /identity/v3/projects/1f731c5517b047d4aca77025cadbb82d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:26:19.886892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934fed-a066-4759-822a-fff8b0568d1a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:19.887497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934fed-a066-4759-822a-fff8b0568d1a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:19.888610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934fed-a066-4759-822a-fff8b0568d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:19.927293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e3934fed-a066-4759-822a-fff8b0568d1a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:19.930304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e3934fed-a066-4759-822a-fff8b0568d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:19.941192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:20.078904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1087/2213] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:20.091486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33779a99-9f42-4e22-8b37-3d4a9612dcef None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:20.092031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33779a99-9f42-4e22-8b37-3d4a9612dcef None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:20.092686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33779a99-9f42-4e22-8b37-3d4a9612dcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:20.128189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-33779a99-9f42-4e22-8b37-3d4a9612dcef None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:20.132547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-33779a99-9f42-4e22-8b37-3d4a9612dcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:20.150709 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1127/2214] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:26:20.155599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3934fed-a066-4759-822a-fff8b0568d1a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:20.177811 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3934fed-a066-4759-822a-fff8b0568d1a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:20.198735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4799a-73b4-4a5f-b896-72eb92c493bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:20.199766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4799a-73b4-4a5f-b896-72eb92c493bc admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:20.200057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4799a-73b4-4a5f-b896-72eb92c493bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:20.200331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4799a-73b4-4a5f-b896-72eb92c493bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:20.201548 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4799a-73b4-4a5f-b896-72eb92c493bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:26:20.201761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4799a-73b4-4a5f-b896-72eb92c493bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:20.209817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4799a-73b4-4a5f-b896-72eb92c493bc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:20.241856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1088/2215] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:26:20.247404 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33779a99-9f42-4e22-8b37-3d4a9612dcef None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:20.258877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33779a99-9f42-4e22-8b37-3d4a9612dcef None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:20.264297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ada294-4846-4be5-abeb-002d2b649fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:20.264990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ada294-4846-4be5-abeb-002d2b649fdb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:20.265317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ada294-4846-4be5-abeb-002d2b649fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:20.265573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ada294-4846-4be5-abeb-002d2b649fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:20.267125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ada294-4846-4be5-abeb-002d2b649fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:20.269598 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ada294-4846-4be5-abeb-002d2b649fdb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:20.319084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1128/2216] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:26:20 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:26:20.325111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4799a-73b4-4a5f-b896-72eb92c493bc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:20.337671 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4799a-73b4-4a5f-b896-72eb92c493bc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:20.447774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9f99a-c046-4038-a67c-1427f6f77016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:20.449441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9f99a-c046-4038-a67c-1427f6f77016 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:20.450216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9f99a-c046-4038-a67c-1427f6f77016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:20.451046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9f99a-c046-4038-a67c-1427f6f77016 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:20.452914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9f99a-c046-4038-a67c-1427f6f77016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:20.458598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9f99a-c046-4038-a67c-1427f6f77016 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:20.511245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1089/2217] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:26:20 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:26:20.517562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ada294-4846-4be5-abeb-002d2b649fdb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:20.543540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ada294-4846-4be5-abeb-002d2b649fdb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:20.563289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5010c-f71a-4b11-a504-855ef47ec8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:20.566530 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5010c-f71a-4b11-a504-855ef47ec8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:20.567516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5010c-f71a-4b11-a504-855ef47ec8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:20.568845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5010c-f71a-4b11-a504-855ef47ec8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:20.570515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5010c-f71a-4b11-a504-855ef47ec8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:20.573636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5010c-f71a-4b11-a504-855ef47ec8d8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:20.598789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1129/2218] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:26:20.602653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9f99a-c046-4038-a67c-1427f6f77016 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:20.616456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9f99a-c046-4038-a67c-1427f6f77016 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:20.623382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ae341-744d-459c-977a-f7908525f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:20.624793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ae341-744d-459c-977a-f7908525f8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:20.625485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ae341-744d-459c-977a-f7908525f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:20.627232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ae341-744d-459c-977a-f7908525f8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:20.629490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ae341-744d-459c-977a-f7908525f8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:20.632073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ae341-744d-459c-977a-f7908525f8ee admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:20.661652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1090/2219] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:26:20.663910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5010c-f71a-4b11-a504-855ef47ec8d8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:20.686809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5010c-f71a-4b11-a504-855ef47ec8d8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:20.696302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8b28c-9c7d-4247-b9c2-6b77d03943eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:20.698057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8b28c-9c7d-4247-b9c2-6b77d03943eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:20.698473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8b28c-9c7d-4247-b9c2-6b77d03943eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:20.698961 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8b28c-9c7d-4247-b9c2-6b77d03943eb admin admin] PATH_INFO: `/v3/projects/40a9c7b67e894fe09b04ab789b32b405/users/e7d45ab3d8e845c98c77da382e0c3e80/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:20.733994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8b28c-9c7d-4247-b9c2-6b77d03943eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a9c7b67e894fe09b04ab789b32b405, user_id=e7d45ab3d8e845c98c77da382e0c3e80, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:20.738216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8b28c-9c7d-4247-b9c2-6b77d03943eb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:20.742847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:20.775854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1130/2220] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:26:20 2022] PUT /identity/v3/projects/40a9c7b67e894fe09b04ab789b32b405/users/e7d45ab3d8e845c98c77da382e0c3e80/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:26:20.788211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509ba68-c002-43d0-a69c-7f6581316c6a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:20.788982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509ba68-c002-43d0-a69c-7f6581316c6a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:20.789729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509ba68-c002-43d0-a69c-7f6581316c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:20.830373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b509ba68-c002-43d0-a69c-7f6581316c6a None None] MFA Rules not processed for user `e7d45ab3d8e845c98c77da382e0c3e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:20.832375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b509ba68-c002-43d0-a69c-7f6581316c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:20.964309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1091/2221] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:26:20.983872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8b28c-9c7d-4247-b9c2-6b77d03943eb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:20.986767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4eb8b28c-9c7d-4247-b9c2-6b77d03943eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:21.113461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8b28c-9c7d-4247-b9c2-6b77d03943eb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:21.119155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720d09b-36ea-4446-9a64-e1aba7d1de24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:21.120469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720d09b-36ea-4446-9a64-e1aba7d1de24 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:21.121003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720d09b-36ea-4446-9a64-e1aba7d1de24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:21.121507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720d09b-36ea-4446-9a64-e1aba7d1de24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:21.132799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720d09b-36ea-4446-9a64-e1aba7d1de24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:21.134719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720d09b-36ea-4446-9a64-e1aba7d1de24 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:21.146709 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1131/2222] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:26:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:26:22.803057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc3f8c-d21d-4630-bcf7-aaeac293689d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:22.803839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc3f8c-d21d-4630-bcf7-aaeac293689d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:22.803839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc3f8c-d21d-4630-bcf7-aaeac293689d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:22.811042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:22.829411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-11dc3f8c-d21d-4630-bcf7-aaeac293689d None None] MFA Rules not processed for user `e7d45ab3d8e845c98c77da382e0c3e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:22.831788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-11dc3f8c-d21d-4630-bcf7-aaeac293689d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:22.842031 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1092/2223] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:26:31.142128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:32.826269 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:36.351369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e67a3-eccb-4f9a-bf21-43249f0da082 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:36.352218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e67a3-eccb-4f9a-bf21-43249f0da082 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:36.352218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e67a3-eccb-4f9a-bf21-43249f0da082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:36.388914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-987e67a3-eccb-4f9a-bf21-43249f0da082 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:36.391273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-987e67a3-eccb-4f9a-bf21-43249f0da082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:36.488700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1132/2224] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 6 14:26:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:36.501296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc3f8c-d21d-4630-bcf7-aaeac293689d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:36.506768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-11dc3f8c-d21d-4630-bcf7-aaeac293689d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:36.607257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc3f8c-d21d-4630-bcf7-aaeac293689d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:36.612724 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47965672-a153-44f8-864f-d32b8229d186 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:36.613444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47965672-a153-44f8-864f-d32b8229d186 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:36.613869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47965672-a153-44f8-864f-d32b8229d186 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:36.614192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47965672-a153-44f8-864f-d32b8229d186 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:36.623800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47965672-a153-44f8-864f-d32b8229d186 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:36.626131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47965672-a153-44f8-864f-d32b8229d186 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:36.636035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1093/2225] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:26:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:26:41.120805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc7d2a-38b3-4ca4-85df-c8264c9c7e10 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:41.122460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc7d2a-38b3-4ca4-85df-c8264c9c7e10 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:41.123087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc7d2a-38b3-4ca4-85df-c8264c9c7e10 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:41.135260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1133/2226] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:26:41 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:26:41.175623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75577152-04b1-4392-8530-ae45e240da36 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:41.176688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75577152-04b1-4392-8530-ae45e240da36 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:41.176688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75577152-04b1-4392-8530-ae45e240da36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:41.237840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-75577152-04b1-4392-8530-ae45e240da36 None None] MFA Rules not processed for user `51b91456a0634e0fa22fb389f0721a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:41.241008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-75577152-04b1-4392-8530-ae45e240da36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:41.250220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc7d2a-38b3-4ca4-85df-c8264c9c7e10 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:41.254345 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f4dc7d2a-38b3-4ca4-85df-c8264c9c7e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:41.351257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1094/2227] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:41.366001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75577152-04b1-4392-8530-ae45e240da36 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:41.367465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-75577152-04b1-4392-8530-ae45e240da36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:41.372122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc7d2a-38b3-4ca4-85df-c8264c9c7e10 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:41.382606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:41.383598 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:41.383769 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:41.384119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] PATH_INFO: `/v3/users/6197b2673c3e41eaac7ca07ad56aab1f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:41.385466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75577152-04b1-4392-8530-ae45e240da36 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:41.385619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6197b2673c3e41eaac7ca07ad56aab1f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:41.387533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:41.396669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f38d1-e3b5-46f9-8738-01833166794b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:41.398047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f38d1-e3b5-46f9-8738-01833166794b service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:41.398682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f38d1-e3b5-46f9-8738-01833166794b service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:41.399266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f38d1-e3b5-46f9-8738-01833166794b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:41.417842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f38d1-e3b5-46f9-8738-01833166794b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:41.421613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f38d1-e3b5-46f9-8738-01833166794b service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:41.448092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1095/2228] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:26:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:26:41.513689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6197b2673c3e41eaac7ca07ad56aab1f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:41.528381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6197b2673c3e41eaac7ca07ad56aab1f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:41.529808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6197b2673c3e41eaac7ca07ad56aab1f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:41.544134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6197b2673c3e41eaac7ca07ad56aab1f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:41.576022 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4c466c80-9641-42ad-8d53-074a0831f4e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6197b2673c3e41eaac7ca07ad56aab1f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:41.584527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1134/2229] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:26:41 2022] DELETE /identity/v3/users/6197b2673c3e41eaac7ca07ad56aab1f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:26:41.815130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f38d1-e3b5-46f9-8738-01833166794b service placement] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:41.835455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f38d1-e3b5-46f9-8738-01833166794b service placement] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:41.924123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:41.925057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:41.925345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:41.925626 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] PATH_INFO: `/v3/projects/87c64a64f6864b84867141833e107e9f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:41.927050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c64a64f6864b84867141833e107e9f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:41.929061 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:41.930905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:42.001420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c64a64f6864b84867141833e107e9f', 'request_id': 'req-15ffbb50-5479-48ef-8c8c-abaa786f83b9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:42.010393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c64a64f6864b84867141833e107e9f', 'request_id': 'req-15ffbb50-5479-48ef-8c8c-abaa786f83b9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:42.019221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c64a64f6864b84867141833e107e9f', 'request_id': 'req-15ffbb50-5479-48ef-8c8c-abaa786f83b9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:42.036542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] The token cache is being invalidate because project 87c64a64f6864b84867141833e107e9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:26:42.036995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:42.038111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-15ffbb50-5479-48ef-8c8c-abaa786f83b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:42.040212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1096/2230] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:26:41 2022] DELETE /identity/v3/projects/87c64a64f6864b84867141833e107e9f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:26:42.046408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034f4d7-d624-4054-a844-16ed6cee8d9b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:42.047166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034f4d7-d624-4054-a844-16ed6cee8d9b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:42.047803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034f4d7-d624-4054-a844-16ed6cee8d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:42.075156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0034f4d7-d624-4054-a844-16ed6cee8d9b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:42.076562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0034f4d7-d624-4054-a844-16ed6cee8d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:42.112941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:42.178621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1135/2231] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:42.187344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6228a91-4777-45d7-bb67-fc3274f0d35b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:42.187961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6228a91-4777-45d7-bb67-fc3274f0d35b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:42.187961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6228a91-4777-45d7-bb67-fc3274f0d35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:42.209808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a6228a91-4777-45d7-bb67-fc3274f0d35b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:42.211192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a6228a91-4777-45d7-bb67-fc3274f0d35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:42.221214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1097/2232] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:26:42.226105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034f4d7-d624-4054-a844-16ed6cee8d9b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:42.238112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034f4d7-d624-4054-a844-16ed6cee8d9b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:42.243395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d87a-ff91-4c06-8935-e35bbf70c11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:42.244217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d87a-ff91-4c06-8935-e35bbf70c11c admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:42.244595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d87a-ff91-4c06-8935-e35bbf70c11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:42.244972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d87a-ff91-4c06-8935-e35bbf70c11c admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:42.245985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d87a-ff91-4c06-8935-e35bbf70c11c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:26:42.246394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d87a-ff91-4c06-8935-e35bbf70c11c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:42.248297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d87a-ff91-4c06-8935-e35bbf70c11c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:42.262487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1136/2233] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:26:42.268262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6228a91-4777-45d7-bb67-fc3274f0d35b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:42.285583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6228a91-4777-45d7-bb67-fc3274f0d35b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:42.292100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b3e76-2004-4240-acb8-20dd3e26d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:42.292873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b3e76-2004-4240-acb8-20dd3e26d5af admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:42.293183 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b3e76-2004-4240-acb8-20dd3e26d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:42.293429 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b3e76-2004-4240-acb8-20dd3e26d5af admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:42.294543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b3e76-2004-4240-acb8-20dd3e26d5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:42.298184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b3e76-2004-4240-acb8-20dd3e26d5af admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:42.337894 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1098/2234] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:26:42 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:26:42.342600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d87a-ff91-4c06-8935-e35bbf70c11c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:42.363726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d87a-ff91-4c06-8935-e35bbf70c11c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:42.467021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495895f-a82a-46e3-8662-b452094ec67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:42.467754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495895f-a82a-46e3-8662-b452094ec67b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:42.468052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495895f-a82a-46e3-8662-b452094ec67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:42.468329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495895f-a82a-46e3-8662-b452094ec67b admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:42.469586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495895f-a82a-46e3-8662-b452094ec67b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:42.471663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495895f-a82a-46e3-8662-b452094ec67b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:42.495555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1137/2235] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:26:42 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:26:42.500973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b3e76-2004-4240-acb8-20dd3e26d5af admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:42.516493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b3e76-2004-4240-acb8-20dd3e26d5af admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:42.521741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffb50f-7908-4e80-a201-dfa3c44f653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:42.522435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffb50f-7908-4e80-a201-dfa3c44f653a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:42.522723 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffb50f-7908-4e80-a201-dfa3c44f653a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:42.523009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffb50f-7908-4e80-a201-dfa3c44f653a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:42.523756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffb50f-7908-4e80-a201-dfa3c44f653a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:42.525438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffb50f-7908-4e80-a201-dfa3c44f653a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:42.542471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1099/2236] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:26:42.549737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495895f-a82a-46e3-8662-b452094ec67b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:42.562277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495895f-a82a-46e3-8662-b452094ec67b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:42.567626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ba508-a604-4dea-9151-71a7c0307fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:42.568360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ba508-a604-4dea-9151-71a7c0307fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:42.568645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ba508-a604-4dea-9151-71a7c0307fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:42.568927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ba508-a604-4dea-9151-71a7c0307fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:42.569695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ba508-a604-4dea-9151-71a7c0307fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:42.571416 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ba508-a604-4dea-9151-71a7c0307fdc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:42.584717 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1138/2237] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:26:42.589990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffb50f-7908-4e80-a201-dfa3c44f653a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:42.613255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffb50f-7908-4e80-a201-dfa3c44f653a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:42.621933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c1fbe-2090-40ce-a4e5-f7a708a7be1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:42.623516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c1fbe-2090-40ce-a4e5-f7a708a7be1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:42.623985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c1fbe-2090-40ce-a4e5-f7a708a7be1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:42.624465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c1fbe-2090-40ce-a4e5-f7a708a7be1f admin admin] PATH_INFO: `/v3/projects/d6e503a8c96e43d4a038fae5275fed83/users/b68bea1c587b4e1c80967bd20844e8ab/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:42.662065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c1fbe-2090-40ce-a4e5-f7a708a7be1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e503a8c96e43d4a038fae5275fed83, user_id=b68bea1c587b4e1c80967bd20844e8ab, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:42.667338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c1fbe-2090-40ce-a4e5-f7a708a7be1f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:42.686709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1100/2238] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:26:42 2022] PUT /identity/v3/projects/d6e503a8c96e43d4a038fae5275fed83/users/b68bea1c587b4e1c80967bd20844e8ab/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:26:42.697737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455ff4e-9d3e-4cac-9f9b-e3958fb45105 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:42.698149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455ff4e-9d3e-4cac-9f9b-e3958fb45105 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:42.698496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455ff4e-9d3e-4cac-9f9b-e3958fb45105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:42.743375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7455ff4e-9d3e-4cac-9f9b-e3958fb45105 None None] MFA Rules not processed for user `b68bea1c587b4e1c80967bd20844e8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:42.745552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7455ff4e-9d3e-4cac-9f9b-e3958fb45105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:42.913047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1139/2239] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:42.931342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c1fbe-2090-40ce-a4e5-f7a708a7be1f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:42.934709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3f2c1fbe-2090-40ce-a4e5-f7a708a7be1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:43.062442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c1fbe-2090-40ce-a4e5-f7a708a7be1f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:43.068092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77179c5c-60d0-4075-a99d-8a8402015368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:43.068769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77179c5c-60d0-4075-a99d-8a8402015368 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:43.069052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77179c5c-60d0-4075-a99d-8a8402015368 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:43.069327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77179c5c-60d0-4075-a99d-8a8402015368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:43.085188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77179c5c-60d0-4075-a99d-8a8402015368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:43.087999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77179c5c-60d0-4075-a99d-8a8402015368 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:43.104359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1101/2240] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:26:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:26:45.946367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455ff4e-9d3e-4cac-9f9b-e3958fb45105 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:46.042169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:46.136112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455ff4e-9d3e-4cac-9f9b-e3958fb45105 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:46.145341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:46.147584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:46.147584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:46.148051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] PATH_INFO: `/v3/users/b68bea1c587b4e1c80967bd20844e8ab` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:46.151060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68bea1c587b4e1c80967bd20844e8ab)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:46.154514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:46.290809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68bea1c587b4e1c80967bd20844e8ab'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:46.308395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68bea1c587b4e1c80967bd20844e8ab'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:46.309591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68bea1c587b4e1c80967bd20844e8ab'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:46.330161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68bea1c587b4e1c80967bd20844e8ab'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:46.356327 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68bea1c587b4e1c80967bd20844e8ab'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:46.369143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1140/2241] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:26:45 2022] DELETE /identity/v3/users/b68bea1c587b4e1c80967bd20844e8ab => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:26:46.373974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4a568-c588-4cc9-97ea-3bd3c784a33e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:46.374551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4a568-c588-4cc9-97ea-3bd3c784a33e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:46.374551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4a568-c588-4cc9-97ea-3bd3c784a33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:46.399251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f2b4a568-c588-4cc9-97ea-3bd3c784a33e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:46.400677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f2b4a568-c588-4cc9-97ea-3bd3c784a33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:46.439034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:46.529208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1102/2242] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:46.539800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:46.541463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:46.639007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4fc83-b0d9-4b0f-8176-c5a69254eb70 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:46.644568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267edfde-224c-43b0-9df9-250470dea04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:46.645505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267edfde-224c-43b0-9df9-250470dea04b service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:46.646031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267edfde-224c-43b0-9df9-250470dea04b service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:46.646732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267edfde-224c-43b0-9df9-250470dea04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:46.669107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267edfde-224c-43b0-9df9-250470dea04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:46.671724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267edfde-224c-43b0-9df9-250470dea04b service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:46.690955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1141/2243] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:26:47.925501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4a568-c588-4cc9-97ea-3bd3c784a33e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:47.928529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f2b4a568-c588-4cc9-97ea-3bd3c784a33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:47.952382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4a568-c588-4cc9-97ea-3bd3c784a33e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:47.963196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:47.966224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:47.967308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:47.968095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] PATH_INFO: `/v3/projects/d6e503a8c96e43d4a038fae5275fed83` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:47.975631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e503a8c96e43d4a038fae5275fed83)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:47.980181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:48.117204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e503a8c96e43d4a038fae5275fed83', 'request_id': 'req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:48.130798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e503a8c96e43d4a038fae5275fed83', 'request_id': 'req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:48.141275 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e503a8c96e43d4a038fae5275fed83', 'request_id': 'req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:48.166453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] The token cache is being invalidate because project d6e503a8c96e43d4a038fae5275fed83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:26:48.166809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:48.173120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:48.176301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1103/2244] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:26:47 2022] DELETE /identity/v3/projects/d6e503a8c96e43d4a038fae5275fed83 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:26:51.262535 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629b04b-3ca7-409f-b0f2-c73f093d6d86 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:51.263173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629b04b-3ca7-409f-b0f2-c73f093d6d86 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:51.263173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629b04b-3ca7-409f-b0f2-c73f093d6d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:51.291538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f629b04b-3ca7-409f-b0f2-c73f093d6d86 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:51.293695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f629b04b-3ca7-409f-b0f2-c73f093d6d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:51.350301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:51.378390 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1142/2245] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:26:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:26:51.398696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:51.400770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:51.506170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da80088-cc3f-45f6-bb5a-cd1cf18bbcd4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:51.518424 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a919da-4252-4aee-82f1-922e59965804 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:51.519648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a919da-4252-4aee-82f1-922e59965804 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:51.520149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a919da-4252-4aee-82f1-922e59965804 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:51.520640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a919da-4252-4aee-82f1-922e59965804 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:51.539529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a919da-4252-4aee-82f1-922e59965804 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:51.542403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a919da-4252-4aee-82f1-922e59965804 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:51.599778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1104/2246] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:26:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:26:52.510251 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78341fc7-ad5a-495c-a286-a068669dd1d1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:52.510904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78341fc7-ad5a-495c-a286-a068669dd1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:52.510904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78341fc7-ad5a-495c-a286-a068669dd1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:52.532827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-78341fc7-ad5a-495c-a286-a068669dd1d1 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:52.535124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-78341fc7-ad5a-495c-a286-a068669dd1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:52.548384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1143/2247] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 6 14:26:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:52.568659 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a919da-4252-4aee-82f1-922e59965804 service swift] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:52.580545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a919da-4252-4aee-82f1-922e59965804 service swift] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:52.586376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d093769-decb-4f85-97b4-6ea17b0b98eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:52.587143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d093769-decb-4f85-97b4-6ea17b0b98eb service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:52.587492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d093769-decb-4f85-97b4-6ea17b0b98eb service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:52.587854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d093769-decb-4f85-97b4-6ea17b0b98eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:52.598038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d093769-decb-4f85-97b4-6ea17b0b98eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:52.599833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d093769-decb-4f85-97b4-6ea17b0b98eb service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:52.611749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1105/2248] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:26:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:26:52.889652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80208d1-b430-4633-9d24-4f2d6842c4ea None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:52.890010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80208d1-b430-4633-9d24-4f2d6842c4ea None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:52.890246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80208d1-b430-4633-9d24-4f2d6842c4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:52.921415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e80208d1-b430-4633-9d24-4f2d6842c4ea None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:52.924192 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e80208d1-b430-4633-9d24-4f2d6842c4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:52.936748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1144/2249] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:26:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:26:52.952679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d093769-decb-4f85-97b4-6ea17b0b98eb service swift] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:52.968338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d093769-decb-4f85-97b4-6ea17b0b98eb service swift] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:52.974678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9a309-c146-48bd-9981-266275538346 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:52.975666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9a309-c146-48bd-9981-266275538346 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:52.975952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9a309-c146-48bd-9981-266275538346 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:52.976224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9a309-c146-48bd-9981-266275538346 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:52.987082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9a309-c146-48bd-9981-266275538346 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:52.991770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9a309-c146-48bd-9981-266275538346 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:52.999963 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1106/2250] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:26:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:26:53.617616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb010f-81a5-461f-b237-0c73d9bb9664 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:53.618263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb010f-81a5-461f-b237-0c73d9bb9664 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:53.618852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb010f-81a5-461f-b237-0c73d9bb9664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:53.641960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-33cb010f-81a5-461f-b237-0c73d9bb9664 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:53.643240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-33cb010f-81a5-461f-b237-0c73d9bb9664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:53.651953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1145/2251] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 6 14:26:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:53.668217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9a309-c146-48bd-9981-266275538346 service swift] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:53.686078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9a309-c146-48bd-9981-266275538346 service swift] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:53.694191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7caf6a-201b-49a9-a4bd-e2f3adae62f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:53.694950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7caf6a-201b-49a9-a4bd-e2f3adae62f6 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:53.695281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7caf6a-201b-49a9-a4bd-e2f3adae62f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:53.695604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7caf6a-201b-49a9-a4bd-e2f3adae62f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:53.706486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7caf6a-201b-49a9-a4bd-e2f3adae62f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:53.708129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7caf6a-201b-49a9-a4bd-e2f3adae62f6 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:53.717917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1107/2252] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:26:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:26:58.514158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb010f-81a5-461f-b237-0c73d9bb9664 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:58.518407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-33cb010f-81a5-461f-b237-0c73d9bb9664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:58.646605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb010f-81a5-461f-b237-0c73d9bb9664 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:58.651879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:58.652877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:58.653206 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:58.653526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] PATH_INFO: `/v3/users/e7d45ab3d8e845c98c77da382e0c3e80` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:58.655069 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d45ab3d8e845c98c77da382e0c3e80)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:58.657470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:58.717848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d45ab3d8e845c98c77da382e0c3e80'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:58.730467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d45ab3d8e845c98c77da382e0c3e80'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:58.730792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d45ab3d8e845c98c77da382e0c3e80'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:58.741172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d45ab3d8e845c98c77da382e0c3e80'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:58.765055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d45ab3d8e845c98c77da382e0c3e80'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:58.777502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1146/2253] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:26:58 2022] DELETE /identity/v3/users/e7d45ab3d8e845c98c77da382e0c3e80 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:26:58.783920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de52164-68e0-49ac-87d2-51d159a9b406 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:58.784735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de52164-68e0-49ac-87d2-51d159a9b406 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:58.784880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de52164-68e0-49ac-87d2-51d159a9b406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:58.792004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:58.817775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5de52164-68e0-49ac-87d2-51d159a9b406 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:58.820008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5de52164-68e0-49ac-87d2-51d159a9b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:58.936309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1108/2254] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:58.948371 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:58.950748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:59.033941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:26:59.043217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71ee42-cb4a-4534-afb7-7d447bcec1bf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:59.051404 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb3158-3e2c-4ba3-93cd-c18e7127b386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:59.052397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb3158-3e2c-4ba3-93cd-c18e7127b386 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:59.052998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb3158-3e2c-4ba3-93cd-c18e7127b386 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:59.053381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb3158-3e2c-4ba3-93cd-c18e7127b386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:59.098442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb3158-3e2c-4ba3-93cd-c18e7127b386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:59.098442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb3158-3e2c-4ba3-93cd-c18e7127b386 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:59.126320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1147/2255] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:26:59.453264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de52164-68e0-49ac-87d2-51d159a9b406 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:59.455074 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5de52164-68e0-49ac-87d2-51d159a9b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:59.472704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de52164-68e0-49ac-87d2-51d159a9b406 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:59.482101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:59.483821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:59.484482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:59.485099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] PATH_INFO: `/v3/projects/40a9c7b67e894fe09b04ab789b32b405` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:59.487797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a9c7b67e894fe09b04ab789b32b405)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:59.492149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:26:59.589135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a9c7b67e894fe09b04ab789b32b405', 'request_id': 'req-34be620d-b8a0-4547-8031-7bf873cb4701'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:59.600945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a9c7b67e894fe09b04ab789b32b405', 'request_id': 'req-34be620d-b8a0-4547-8031-7bf873cb4701'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:59.626856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a9c7b67e894fe09b04ab789b32b405', 'request_id': 'req-34be620d-b8a0-4547-8031-7bf873cb4701'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:59.658688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] The token cache is being invalidate because project 40a9c7b67e894fe09b04ab789b32b405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:26:59.658979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:59.661602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34be620d-b8a0-4547-8031-7bf873cb4701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:26:59.666682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1109/2256] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:26:59 2022] DELETE /identity/v3/projects/40a9c7b67e894fe09b04ab789b32b405 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:26:59.677308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647edcb7-e3bc-4b83-b008-a88d49ffb2b7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:59.677308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647edcb7-e3bc-4b83-b008-a88d49ffb2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:59.678001 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647edcb7-e3bc-4b83-b008-a88d49ffb2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:59.721696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-647edcb7-e3bc-4b83-b008-a88d49ffb2b7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:59.723166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-647edcb7-e3bc-4b83-b008-a88d49ffb2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:59.842979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1148/2257] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:59.852941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8a15f-14dd-4206-ae12-1f209f682117 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:59.853517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8a15f-14dd-4206-ae12-1f209f682117 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:59.853897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8a15f-14dd-4206-ae12-1f209f682117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:59.894068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a4f8a15f-14dd-4206-ae12-1f209f682117 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:26:59.895774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a4f8a15f-14dd-4206-ae12-1f209f682117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:26:59.927435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1110/2258] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:26:59.928790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647edcb7-e3bc-4b83-b008-a88d49ffb2b7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:26:59.963313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647edcb7-e3bc-4b83-b008-a88d49ffb2b7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:26:59.970440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b7ef2-d767-4f9d-b1d7-57cbd73c9fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:26:59.971404 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b7ef2-d767-4f9d-b1d7-57cbd73c9fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:26:59.971792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b7ef2-d767-4f9d-b1d7-57cbd73c9fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:26:59.972168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b7ef2-d767-4f9d-b1d7-57cbd73c9fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:26:59.973095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b7ef2-d767-4f9d-b1d7-57cbd73c9fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:26:59.973577 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b7ef2-d767-4f9d-b1d7-57cbd73c9fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:26:59.975917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b7ef2-d767-4f9d-b1d7-57cbd73c9fd9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:00.002930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1149/2259] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:27:00.007306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8a15f-14dd-4206-ae12-1f209f682117 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:00.020479 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8a15f-14dd-4206-ae12-1f209f682117 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:00.026255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983252d-3bb6-4cbe-8723-5d02e0c8827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:00.026970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983252d-3bb6-4cbe-8723-5d02e0c8827a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:00.027465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983252d-3bb6-4cbe-8723-5d02e0c8827a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:00.027738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983252d-3bb6-4cbe-8723-5d02e0c8827a admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:00.028715 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983252d-3bb6-4cbe-8723-5d02e0c8827a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:00.031776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983252d-3bb6-4cbe-8723-5d02e0c8827a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:00.076088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1111/2260] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:27:00 2022] POST /identity/v3/projects => generated 397 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:27:00.082634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b7ef2-d767-4f9d-b1d7-57cbd73c9fd9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:00.104383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b7ef2-d767-4f9d-b1d7-57cbd73c9fd9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:00.213994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001b26a-1f97-4326-a1b8-4e2f5ec15c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:00.215166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001b26a-1f97-4326-a1b8-4e2f5ec15c91 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:00.215672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001b26a-1f97-4326-a1b8-4e2f5ec15c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:00.216145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001b26a-1f97-4326-a1b8-4e2f5ec15c91 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:00.217005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001b26a-1f97-4326-a1b8-4e2f5ec15c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:00.220706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001b26a-1f97-4326-a1b8-4e2f5ec15c91 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:00.250320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1150/2261] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:27:00 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:27:00.252648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983252d-3bb6-4cbe-8723-5d02e0c8827a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:00.273160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983252d-3bb6-4cbe-8723-5d02e0c8827a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:00.284791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fd85e-a92d-420f-a05e-ccc4400fbe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:00.286832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fd85e-a92d-420f-a05e-ccc4400fbe28 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:00.287630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fd85e-a92d-420f-a05e-ccc4400fbe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:00.288565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fd85e-a92d-420f-a05e-ccc4400fbe28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:00.290419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fd85e-a92d-420f-a05e-ccc4400fbe28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:00.294423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0fd85e-a92d-420f-a05e-ccc4400fbe28 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:00.320543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1112/2262] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:27:00.326753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001b26a-1f97-4326-a1b8-4e2f5ec15c91 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:00.340255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001b26a-1f97-4326-a1b8-4e2f5ec15c91 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:00.346525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe153d88-8725-41ab-a9ec-d27df510e38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:00.347230 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe153d88-8725-41ab-a9ec-d27df510e38a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:00.347513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe153d88-8725-41ab-a9ec-d27df510e38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:00.347792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe153d88-8725-41ab-a9ec-d27df510e38a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:00.348564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe153d88-8725-41ab-a9ec-d27df510e38a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:00.350360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe153d88-8725-41ab-a9ec-d27df510e38a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:00.365167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1151/2263] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:27:00.370072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fd85e-a92d-420f-a05e-ccc4400fbe28 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:00.388633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fd85e-a92d-420f-a05e-ccc4400fbe28 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:00.398024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3afe65-73ed-4d16-b950-4f52b818ef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:00.399549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3afe65-73ed-4d16-b950-4f52b818ef69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:00.400057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3afe65-73ed-4d16-b950-4f52b818ef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:00.400588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3afe65-73ed-4d16-b950-4f52b818ef69 admin admin] PATH_INFO: `/v3/projects/9f77ef2266c04d28bb263ef284b8483c/users/59f432b65c4244fb84bd7220248cec85/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:00.424481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3afe65-73ed-4d16-b950-4f52b818ef69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f77ef2266c04d28bb263ef284b8483c, user_id=59f432b65c4244fb84bd7220248cec85, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:00.427413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3afe65-73ed-4d16-b950-4f52b818ef69 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:00.450945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1113/2264] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:27:00 2022] PUT /identity/v3/projects/9f77ef2266c04d28bb263ef284b8483c/users/59f432b65c4244fb84bd7220248cec85/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:27:00.559421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1062a-8dc2-4ec5-90d3-eba98242618b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:00.559917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1062a-8dc2-4ec5-90d3-eba98242618b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:00.560454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1062a-8dc2-4ec5-90d3-eba98242618b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:00.610853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0bc1062a-8dc2-4ec5-90d3-eba98242618b None None] MFA Rules not processed for user `59f432b65c4244fb84bd7220248cec85`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:00.613155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0bc1062a-8dc2-4ec5-90d3-eba98242618b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:00.690854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:00.725323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1152/2265] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:27:00.742351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3afe65-73ed-4d16-b950-4f52b818ef69 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:00.744892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-db3afe65-73ed-4d16-b950-4f52b818ef69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:00.787604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:00.852291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3afe65-73ed-4d16-b950-4f52b818ef69 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:00.860059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a9f35-f028-4126-a2ac-0e8132ad4ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:00.861759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9f35-f028-4126-a2ac-0e8132ad4ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:00.862438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9f35-f028-4126-a2ac-0e8132ad4ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:00.862961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9f35-f028-4126-a2ac-0e8132ad4ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:00.881536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a9f35-f028-4126-a2ac-0e8132ad4ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:00.884444 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a9f35-f028-4126-a2ac-0e8132ad4ab4 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:00.903207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1114/2266] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:27:02.580752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121e437-c1aa-42b9-9985-f9f9fc5903a4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:02.581599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121e437-c1aa-42b9-9985-f9f9fc5903a4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:02.581836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121e437-c1aa-42b9-9985-f9f9fc5903a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:02.605542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9121e437-c1aa-42b9-9985-f9f9fc5903a4 None None] MFA Rules not processed for user `53505ac8ab6142a6bf0c3d6fe1cc721f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:02.607611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9121e437-c1aa-42b9-9985-f9f9fc5903a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:02.693925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1153/2267] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:27:10.895250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=16 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:12.064226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a9f35-f028-4126-a2ac-0e8132ad4ab4 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:12.141131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a9f35-f028-4126-a2ac-0e8132ad4ab4 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:12.146660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:12.147577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:12.147864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:12.148139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] PATH_INFO: `/v3/users/59f432b65c4244fb84bd7220248cec85` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:12.149611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59f432b65c4244fb84bd7220248cec85)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:12.151625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:12.239784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59f432b65c4244fb84bd7220248cec85'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:12.256945 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59f432b65c4244fb84bd7220248cec85'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:12.258303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59f432b65c4244fb84bd7220248cec85'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:12.286611 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59f432b65c4244fb84bd7220248cec85'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:12.288197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59f432b65c4244fb84bd7220248cec85'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:12.314816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1115/2268] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:27:12 2022] DELETE /identity/v3/users/59f432b65c4244fb84bd7220248cec85 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:27:12.320314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61237a1-1199-47fe-b7f2-35124992c0bb None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:12.321069 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61237a1-1199-47fe-b7f2-35124992c0bb None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:12.321196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61237a1-1199-47fe-b7f2-35124992c0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:12.346322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b61237a1-1199-47fe-b7f2-35124992c0bb None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:12.347713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b61237a1-1199-47fe-b7f2-35124992c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:12.430375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1154/2269] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:27:12.441379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:12.442845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:12.510624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f6f1e-c4b1-4aa9-a7bf-7f20ae3b2151 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:12.515466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a6cc5-159a-42cf-9d1b-e64d30f3e762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:12.516164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a6cc5-159a-42cf-9d1b-e64d30f3e762 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:12.516494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a6cc5-159a-42cf-9d1b-e64d30f3e762 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:12.516702 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a6cc5-159a-42cf-9d1b-e64d30f3e762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:12.526918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a6cc5-159a-42cf-9d1b-e64d30f3e762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:12.528410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a6cc5-159a-42cf-9d1b-e64d30f3e762 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:12.540202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1116/2270] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:12.897861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61237a1-1199-47fe-b7f2-35124992c0bb None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:12.899537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b61237a1-1199-47fe-b7f2-35124992c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:12.922645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61237a1-1199-47fe-b7f2-35124992c0bb None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:12.928283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:12.930047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:12.930218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:12.930401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] PATH_INFO: `/v3/projects/9f77ef2266c04d28bb263ef284b8483c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:12.934628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f77ef2266c04d28bb263ef284b8483c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:12.937939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:12.996514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f77ef2266c04d28bb263ef284b8483c', 'request_id': 'req-7ba5064e-e07c-4c30-9dc6-a97d7e181973'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:13.006817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f77ef2266c04d28bb263ef284b8483c', 'request_id': 'req-7ba5064e-e07c-4c30-9dc6-a97d7e181973'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:13.016981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f77ef2266c04d28bb263ef284b8483c', 'request_id': 'req-7ba5064e-e07c-4c30-9dc6-a97d7e181973'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:13.035783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] The token cache is being invalidate because project 9f77ef2266c04d28bb263ef284b8483c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:27:13.036392 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:13.037633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:13.040179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1155/2271] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:27:12 2022] DELETE /identity/v3/projects/9f77ef2266c04d28bb263ef284b8483c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:27:14.290054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a6cc5-159a-42cf-9d1b-e64d30f3e762 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:14.300406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a6cc5-159a-42cf-9d1b-e64d30f3e762 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:14.322908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:14.376394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:14.377500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:14.378039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:14.378534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:14.381743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:14.521602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:14.524496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:14.543235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1117/2272] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:15.806526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:15.807954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:15.825587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5064e-e07c-4c30-9dc6-a97d7e181973 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:15.831997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73a4f2-9f8d-45a9-97ef-59dfcb46ded0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:15.833107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73a4f2-9f8d-45a9-97ef-59dfcb46ded0 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:15.833752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73a4f2-9f8d-45a9-97ef-59dfcb46ded0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:15.834463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73a4f2-9f8d-45a9-97ef-59dfcb46ded0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:15.838050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bc73a4f2-9f8d-45a9-97ef-59dfcb46ded0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:15.861859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73a4f2-9f8d-45a9-97ef-59dfcb46ded0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:15.863736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73a4f2-9f8d-45a9-97ef-59dfcb46ded0 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:15.873547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:15.898016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1156/2273] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:27:19.232049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:19.235067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:19.366003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb6ed7-8239-4926-bee0-f2f09a303018 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:19.374351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:19.375856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:19.376334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:19.376821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] PATH_INFO: `/v3/users/66ded9372b644c57a9f13fc88ff7fdb6` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:19.379099 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ded9372b644c57a9f13fc88ff7fdb6)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:19.381394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:19.457867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ded9372b644c57a9f13fc88ff7fdb6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:19.474228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ded9372b644c57a9f13fc88ff7fdb6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:19.475631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ded9372b644c57a9f13fc88ff7fdb6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:19.492101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:19.504636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ded9372b644c57a9f13fc88ff7fdb6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:19.505800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d2716395-b88c-4cb3-bd8e-4f554c2a9ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ded9372b644c57a9f13fc88ff7fdb6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:19.531817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1118/2274] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:27:19 2022] DELETE /identity/v3/users/66ded9372b644c57a9f13fc88ff7fdb6 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:27:19.782485 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73a4f2-9f8d-45a9-97ef-59dfcb46ded0 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:19.817444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73a4f2-9f8d-45a9-97ef-59dfcb46ded0 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:19.941229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:19.942552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:19.943027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:19.943469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] PATH_INFO: `/v3/projects/810c087aef3d453cb9c0365467dd9134` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:19.945041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] RBAC: Authorizing `identity:delete_project(project_id=810c087aef3d453cb9c0365467dd9134)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:19.947405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:20.033646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810c087aef3d453cb9c0365467dd9134', 'request_id': 'req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:20.043988 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810c087aef3d453cb9c0365467dd9134', 'request_id': 'req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:20.053544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810c087aef3d453cb9c0365467dd9134', 'request_id': 'req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:20.071187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] The token cache is being invalidate because project 810c087aef3d453cb9c0365467dd9134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:27:20.071462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:20.072253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03372c3c-3a8c-4a1c-8e55-a5d99846bfda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:27:20.074037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1157/2275] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:27:19 2022] DELETE /identity/v3/projects/810c087aef3d453cb9c0365467dd9134 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:27:20.080873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2a65f-4053-4a6d-bcf9-71231edc0b49 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:20.081156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2a65f-4053-4a6d-bcf9-71231edc0b49 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:20.081428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2a65f-4053-4a6d-bcf9-71231edc0b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:20.103519 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-45a2a65f-4053-4a6d-bcf9-71231edc0b49 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:20.105027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-45a2a65f-4053-4a6d-bcf9-71231edc0b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:20.184889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1119/2276] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:27:20.194094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc6394-9391-435a-af45-49e2ac485639 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:20.194406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc6394-9391-435a-af45-49e2ac485639 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:20.194664 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc6394-9391-435a-af45-49e2ac485639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:20.222481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5cbc6394-9391-435a-af45-49e2ac485639 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:20.224334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5cbc6394-9391-435a-af45-49e2ac485639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:20.235863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1158/2277] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:27:20.242786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2a65f-4053-4a6d-bcf9-71231edc0b49 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:20.257511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2a65f-4053-4a6d-bcf9-71231edc0b49 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:20.263511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d963-dbd8-418f-b74c-84b41ca633fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:20.265299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d963-dbd8-418f-b74c-84b41ca633fe admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:20.266324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d963-dbd8-418f-b74c-84b41ca633fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:20.266617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d963-dbd8-418f-b74c-84b41ca633fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:20.267420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514d963-dbd8-418f-b74c-84b41ca633fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:27:20.267732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514d963-dbd8-418f-b74c-84b41ca633fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:20.269407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514d963-dbd8-418f-b74c-84b41ca633fe admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:20.285130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1120/2278] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:27:20.290539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc6394-9391-435a-af45-49e2ac485639 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:20.308733 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc6394-9391-435a-af45-49e2ac485639 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:20.316942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820352f3-6a67-480c-962f-bf0a0712882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:20.317985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820352f3-6a67-480c-962f-bf0a0712882d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:20.318387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820352f3-6a67-480c-962f-bf0a0712882d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:20.318840 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820352f3-6a67-480c-962f-bf0a0712882d admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:20.320080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820352f3-6a67-480c-962f-bf0a0712882d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:20.324088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820352f3-6a67-480c-962f-bf0a0712882d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:20.369653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1159/2279] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:27:20 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:27:20.374901 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d963-dbd8-418f-b74c-84b41ca633fe admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:20.395340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d963-dbd8-418f-b74c-84b41ca633fe admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:20.513034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d3370-b2ee-470d-86ee-78d88ebea0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:20.514130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d3370-b2ee-470d-86ee-78d88ebea0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:20.515158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d3370-b2ee-470d-86ee-78d88ebea0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:20.516717 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d3370-b2ee-470d-86ee-78d88ebea0ec admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:20.517533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d3370-b2ee-470d-86ee-78d88ebea0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:20.519704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d3370-b2ee-470d-86ee-78d88ebea0ec admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:20.553340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1121/2280] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:27:20 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:27:20.558151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820352f3-6a67-480c-962f-bf0a0712882d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:20.571674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820352f3-6a67-480c-962f-bf0a0712882d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:20.577104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07d9b-8250-4caf-9a8b-d6baf0e468c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:20.577783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07d9b-8250-4caf-9a8b-d6baf0e468c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:20.578132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07d9b-8250-4caf-9a8b-d6baf0e468c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:20.578444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07d9b-8250-4caf-9a8b-d6baf0e468c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:20.579209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07d9b-8250-4caf-9a8b-d6baf0e468c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:20.580930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07d9b-8250-4caf-9a8b-d6baf0e468c7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:20.594182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1160/2281] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:27:20.598219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d3370-b2ee-470d-86ee-78d88ebea0ec admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:20.611088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d3370-b2ee-470d-86ee-78d88ebea0ec admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:20.616602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fb396-298e-4187-b62c-fc8f3e367cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:20.617301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fb396-298e-4187-b62c-fc8f3e367cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:20.617594 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fb396-298e-4187-b62c-fc8f3e367cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:20.617910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fb396-298e-4187-b62c-fc8f3e367cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:20.618661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fb396-298e-4187-b62c-fc8f3e367cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:20.620330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fb396-298e-4187-b62c-fc8f3e367cc6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:20.634149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1122/2282] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:27:20.639534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07d9b-8250-4caf-9a8b-d6baf0e468c7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:20.656347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07d9b-8250-4caf-9a8b-d6baf0e468c7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:20.663740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d771634-3de8-4ed5-8b18-b9b37b497305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:20.665337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d771634-3de8-4ed5-8b18-b9b37b497305 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:20.665653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d771634-3de8-4ed5-8b18-b9b37b497305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:20.667469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d771634-3de8-4ed5-8b18-b9b37b497305 admin admin] PATH_INFO: `/v3/projects/d1af47083c644609a4a84b1273a4087f/users/791369c0e2574cdf8047a5f178a3c6db/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:20.708933 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d771634-3de8-4ed5-8b18-b9b37b497305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1af47083c644609a4a84b1273a4087f, user_id=791369c0e2574cdf8047a5f178a3c6db, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:20.711284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d771634-3de8-4ed5-8b18-b9b37b497305 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:20.733826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1161/2283] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:27:20 2022] PUT /identity/v3/projects/d1af47083c644609a4a84b1273a4087f/users/791369c0e2574cdf8047a5f178a3c6db/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:27:20.740038 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6e046-f095-4b62-b893-0852b5a5f1fa None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:20.740356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6e046-f095-4b62-b893-0852b5a5f1fa None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:20.740797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6e046-f095-4b62-b893-0852b5a5f1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:20.772027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6af6e046-f095-4b62-b893-0852b5a5f1fa None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:20.774294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6af6e046-f095-4b62-b893-0852b5a5f1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:20.878087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:20.901399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1123/2284] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:27:20.911835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d771634-3de8-4ed5-8b18-b9b37b497305 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:20.913687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6d771634-3de8-4ed5-8b18-b9b37b497305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:20.969188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:21.037600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d771634-3de8-4ed5-8b18-b9b37b497305 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:21.044162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de626d72-7607-4dfd-a34c-562d34cc8b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:21.045360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de626d72-7607-4dfd-a34c-562d34cc8b65 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:21.045944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de626d72-7607-4dfd-a34c-562d34cc8b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:21.046458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de626d72-7607-4dfd-a34c-562d34cc8b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:21.064630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de626d72-7607-4dfd-a34c-562d34cc8b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:21.067602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de626d72-7607-4dfd-a34c-562d34cc8b65 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:21.088286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1162/2285] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:27:29.725252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0ab8-038c-43b1-8c3a-5a822c11a720 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:29.726464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0ab8-038c-43b1-8c3a-5a822c11a720 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:29.726973 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0ab8-038c-43b1-8c3a-5a822c11a720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:29.763188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-889b0ab8-038c-43b1-8c3a-5a822c11a720 None None] MFA Rules not processed for user `791369c0e2574cdf8047a5f178a3c6db`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:29.765456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-889b0ab8-038c-43b1-8c3a-5a822c11a720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:29.849912 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1124/2286] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:27:29.855724 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de626d72-7607-4dfd-a34c-562d34cc8b65 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:29.869814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de626d72-7607-4dfd-a34c-562d34cc8b65 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:29.876875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57f1e1-f708-41f1-9d7f-aa88c25344e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:29.877661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57f1e1-f708-41f1-9d7f-aa88c25344e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:29.877994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57f1e1-f708-41f1-9d7f-aa88c25344e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:29.878276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57f1e1-f708-41f1-9d7f-aa88c25344e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:29.879015 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57f1e1-f708-41f1-9d7f-aa88c25344e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:29.881831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57f1e1-f708-41f1-9d7f-aa88c25344e4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:29.923435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1163/2287] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:27:29 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:27:29.928410 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b0ab8-038c-43b1-8c3a-5a822c11a720 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:29.943441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b0ab8-038c-43b1-8c3a-5a822c11a720 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:29.995202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed51f-fa18-427f-a7dc-46cbe9c682a7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:29.995474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed51f-fa18-427f-a7dc-46cbe9c682a7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:29.996017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed51f-fa18-427f-a7dc-46cbe9c682a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:30.018778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6d2ed51f-fa18-427f-a7dc-46cbe9c682a7 None None] MFA Rules not processed for user `405ac2f6c07545efaa818e10fc788c05`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:30.020032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6d2ed51f-fa18-427f-a7dc-46cbe9c682a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:30.031893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2d07a-7e13-492b-a38b-160cd693af15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:30.032662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2d07a-7e13-492b-a38b-160cd693af15 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:30.032951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2d07a-7e13-492b-a38b-160cd693af15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:30.033237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2d07a-7e13-492b-a38b-160cd693af15 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:30.034052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2d07a-7e13-492b-a38b-160cd693af15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:30.036153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2d07a-7e13-492b-a38b-160cd693af15 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:30.073186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1125/2288] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:27:29 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:27:30.077936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2d07a-7e13-492b-a38b-160cd693af15 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:30.096966 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2d07a-7e13-492b-a38b-160cd693af15 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:30.103507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81ce95-19e1-4088-a387-32a65e3f216f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:30.103507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81ce95-19e1-4088-a387-32a65e3f216f admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:30.103905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81ce95-19e1-4088-a387-32a65e3f216f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:30.104058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81ce95-19e1-4088-a387-32a65e3f216f admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:30.105001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81ce95-19e1-4088-a387-32a65e3f216f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:30.108252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81ce95-19e1-4088-a387-32a65e3f216f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:30.126296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1126/2289] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:27:30.127278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81ce95-19e1-4088-a387-32a65e3f216f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:30.144652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1164/2290] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:27:30.152388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81ce95-19e1-4088-a387-32a65e3f216f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:30.160699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a9ad3-eec0-4c83-ba19-3ac4f1b5b2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:30.162242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a9ad3-eec0-4c83-ba19-3ac4f1b5b2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:30.162712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a9ad3-eec0-4c83-ba19-3ac4f1b5b2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:30.163425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a9ad3-eec0-4c83-ba19-3ac4f1b5b2bb admin admin] PATH_INFO: `/v3/projects/f97acc4269294ff4be07755871d3bb7b/users/8ca409fc0da648a28077a3b34465326b/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:30.197169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a9ad3-eec0-4c83-ba19-3ac4f1b5b2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97acc4269294ff4be07755871d3bb7b, user_id=8ca409fc0da648a28077a3b34465326b, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:30.203628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a9ad3-eec0-4c83-ba19-3ac4f1b5b2bb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:30.226076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1127/2291] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:27:30 2022] PUT /identity/v3/projects/f97acc4269294ff4be07755871d3bb7b/users/8ca409fc0da648a28077a3b34465326b/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:27:30.231998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed51f-fa18-427f-a7dc-46cbe9c682a7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:30.252325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed51f-fa18-427f-a7dc-46cbe9c682a7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:30.375918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e4dcc-1f93-42c5-97f5-b47fca3d6dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:30.376887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e4dcc-1f93-42c5-97f5-b47fca3d6dce admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:30.377383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e4dcc-1f93-42c5-97f5-b47fca3d6dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:30.377945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e4dcc-1f93-42c5-97f5-b47fca3d6dce admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:30.379529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e4dcc-1f93-42c5-97f5-b47fca3d6dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:30.382986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e4dcc-1f93-42c5-97f5-b47fca3d6dce admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:30.407298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1165/2292] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:27:30.411689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a9ad3-eec0-4c83-ba19-3ac4f1b5b2bb admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:30.426995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a9ad3-eec0-4c83-ba19-3ac4f1b5b2bb admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:30.439076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a6479-2d72-4f39-be8a-6b329d7abd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:30.440664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a6479-2d72-4f39-be8a-6b329d7abd60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:30.441279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a6479-2d72-4f39-be8a-6b329d7abd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:30.441609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a6479-2d72-4f39-be8a-6b329d7abd60 admin admin] PATH_INFO: `/v3/projects/f97acc4269294ff4be07755871d3bb7b/users/8ca409fc0da648a28077a3b34465326b/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:30.445379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a6479-2d72-4f39-be8a-6b329d7abd60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97acc4269294ff4be07755871d3bb7b, user_id=8ca409fc0da648a28077a3b34465326b, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:30.450934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a6479-2d72-4f39-be8a-6b329d7abd60 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:30.494073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1128/2293] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:27:30 2022] PUT /identity/v3/projects/f97acc4269294ff4be07755871d3bb7b/users/8ca409fc0da648a28077a3b34465326b/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:27:31.377626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e4dcc-1f93-42c5-97f5-b47fca3d6dce admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:31.384152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a32e4dcc-1f93-42c5-97f5-b47fca3d6dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:31.547649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:31.626333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e4dcc-1f93-42c5-97f5-b47fca3d6dce admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:31.801928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe9a93-8736-409d-8e19-77c851b0e6fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:31.803139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe9a93-8736-409d-8e19-77c851b0e6fc service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:31.803645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe9a93-8736-409d-8e19-77c851b0e6fc service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:31.804141 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe9a93-8736-409d-8e19-77c851b0e6fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:31.810168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-88fe9a93-8736-409d-8e19-77c851b0e6fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:31.842152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe9a93-8736-409d-8e19-77c851b0e6fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:31.845866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe9a93-8736-409d-8e19-77c851b0e6fc service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:31.871906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1166/2294] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:32.367742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3b08b-484e-4ef8-aeee-0da96c6d429b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:32.368649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3b08b-484e-4ef8-aeee-0da96c6d429b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:32.368649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3b08b-484e-4ef8-aeee-0da96c6d429b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:32.408657 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-87a3b08b-484e-4ef8-aeee-0da96c6d429b None None] MFA Rules not processed for user `86f9bcac527c45d6bc81b70e26544924`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:32.409964 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87a3b08b-484e-4ef8-aeee-0da96c6d429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:32.446775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:32.580421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1129/2295] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:27:33.303514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe9a93-8736-409d-8e19-77c851b0e6fc service placement] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:33.478246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe9a93-8736-409d-8e19-77c851b0e6fc service placement] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:33.503267 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:33.504671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:33.505180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:33.505681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:33.513426 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:33.581404 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:33.586313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:33.631723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1167/2296] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:39.869406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3b08b-484e-4ef8-aeee-0da96c6d429b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:39.871516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87a3b08b-484e-4ef8-aeee-0da96c6d429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:39.997217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3b08b-484e-4ef8-aeee-0da96c6d429b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:40.011488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1da282-9743-4e1f-8a2d-79f2222c416e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:40.012445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1da282-9743-4e1f-8a2d-79f2222c416e service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:40.015219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1da282-9743-4e1f-8a2d-79f2222c416e service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:40.015219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1da282-9743-4e1f-8a2d-79f2222c416e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:40.017773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3c1da282-9743-4e1f-8a2d-79f2222c416e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:40.234087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1da282-9743-4e1f-8a2d-79f2222c416e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:40.240458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1da282-9743-4e1f-8a2d-79f2222c416e service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:40.268154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1130/2297] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:27:40.460740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:40.465216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:40.518384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084ec4a-49c0-4ceb-acbf-2d5787264748 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:40.518866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084ec4a-49c0-4ceb-acbf-2d5787264748 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:40.519329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084ec4a-49c0-4ceb-acbf-2d5787264748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:40.625536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3833a1-ac8f-41df-bd52-963e12334aa5 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:40.634301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:40.635509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:40.636272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:40.636784 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:40.640566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:40.674345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e084ec4a-49c0-4ceb-acbf-2d5787264748 None None] MFA Rules not processed for user `8ca409fc0da648a28077a3b34465326b`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:40.679430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e084ec4a-49c0-4ceb-acbf-2d5787264748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:40.863329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:40.885270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:40.937099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:40.940149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:40.961774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1168/2298] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:40.980687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1131/2299] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:27:40.992184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:40.996467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:41.220377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3f532-735b-45b6-b323-fddc2282e40a service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:41.230959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525f7ab-52f9-4651-9a14-4750c39f5aab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41a357957bc47739b7bd6f21d159d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:41.231989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525f7ab-52f9-4651-9a14-4750c39f5aab service glance] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:41.233188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525f7ab-52f9-4651-9a14-4750c39f5aab service glance] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:41.237489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525f7ab-52f9-4651-9a14-4750c39f5aab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:41.371324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525f7ab-52f9-4651-9a14-4750c39f5aab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:41.372882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525f7ab-52f9-4651-9a14-4750c39f5aab service glance] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:41.385123 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1169/2300] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:41.688799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d2683-54ed-4aa6-90b0-d650f9616b00 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:41.689675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d2683-54ed-4aa6-90b0-d650f9616b00 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:41.690030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d2683-54ed-4aa6-90b0-d650f9616b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:41.737656 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-776d2683-54ed-4aa6-90b0-d650f9616b00 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:41.741145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-776d2683-54ed-4aa6-90b0-d650f9616b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:41.864273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1132/2301] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:27:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:27:41.899561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525f7ab-52f9-4651-9a14-4750c39f5aab service glance] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:41.901377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e525f7ab-52f9-4651-9a14-4750c39f5aab service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:42.020460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525f7ab-52f9-4651-9a14-4750c39f5aab service glance] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:42.033398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0f205-b2a6-4815-9a0b-2eca540d152f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:42.034621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0f205-b2a6-4815-9a0b-2eca540d152f service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:42.035113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0f205-b2a6-4815-9a0b-2eca540d152f service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:42.036913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0f205-b2a6-4815-9a0b-2eca540d152f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:42.049593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0f205-b2a6-4815-9a0b-2eca540d152f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:42.052135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0f205-b2a6-4815-9a0b-2eca540d152f service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:42.067729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1170/2302] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:27:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 14:27:42.862452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d2683-54ed-4aa6-90b0-d650f9616b00 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:42.864417 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-776d2683-54ed-4aa6-90b0-d650f9616b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:42.919559 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d2683-54ed-4aa6-90b0-d650f9616b00 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:42.944555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:42.945734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:42.946241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:42.946739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:42.963786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:43.013403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:43.017125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:43.070994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1133/2303] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:52.062094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:53.038997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=37 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:27:53.250011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da22e06-c203-4153-a130-874dbc4fb9ad None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:53.250936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da22e06-c203-4153-a130-874dbc4fb9ad None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:53.251489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da22e06-c203-4153-a130-874dbc4fb9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:53.290803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0da22e06-c203-4153-a130-874dbc4fb9ad None None] MFA Rules not processed for user `3f2e1c5d022c4bde9e76b8a933739557`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:53.292551 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0da22e06-c203-4153-a130-874dbc4fb9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:53.314719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1171/2304] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:27:54.030630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:54.033677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:54.054757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7b11e-7783-41fb-a71d-e93a56d39ec0 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:54.063518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b08ce2-27e9-4657-b0a4-b083e1de6478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:54.064361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b08ce2-27e9-4657-b0a4-b083e1de6478 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:54.064737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b08ce2-27e9-4657-b0a4-b083e1de6478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:54.065113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b08ce2-27e9-4657-b0a4-b083e1de6478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:54.068580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-69b08ce2-27e9-4657-b0a4-b083e1de6478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:54.107226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b08ce2-27e9-4657-b0a4-b083e1de6478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:54.110137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b08ce2-27e9-4657-b0a4-b083e1de6478 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:54.134994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1134/2305] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:56.097350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02f914-7132-474b-9948-ce596a4b84e0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:56.098248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02f914-7132-474b-9948-ce596a4b84e0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:56.098248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02f914-7132-474b-9948-ce596a4b84e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:56.119706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6d02f914-7132-474b-9948-ce596a4b84e0 None None] MFA Rules not processed for user `791369c0e2574cdf8047a5f178a3c6db`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:56.121060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6d02f914-7132-474b-9948-ce596a4b84e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:56.129617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1172/2306] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:27:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:27:56.583652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b08ce2-27e9-4657-b0a4-b083e1de6478 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:56.594821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b08ce2-27e9-4657-b0a4-b083e1de6478 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:56.599425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0341ec-83da-4708-a0c7-81e133f0bb0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:56.600330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0341ec-83da-4708-a0c7-81e133f0bb0f service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:56.600754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0341ec-83da-4708-a0c7-81e133f0bb0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:56.601126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0341ec-83da-4708-a0c7-81e133f0bb0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:56.603502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-df0341ec-83da-4708-a0c7-81e133f0bb0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:56.616317 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0341ec-83da-4708-a0c7-81e133f0bb0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:56.617812 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0341ec-83da-4708-a0c7-81e133f0bb0f service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:56.635087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1135/2307] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:57.558918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02f914-7132-474b-9948-ce596a4b84e0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:57.586056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02f914-7132-474b-9948-ce596a4b84e0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:57.597805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d874bb-8c20-406d-9b80-19b8680ea8ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:57.599487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d874bb-8c20-406d-9b80-19b8680ea8ed service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:57.600280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d874bb-8c20-406d-9b80-19b8680ea8ed service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:57.600849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d874bb-8c20-406d-9b80-19b8680ea8ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:57.604314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d6d874bb-8c20-406d-9b80-19b8680ea8ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:57.728197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d874bb-8c20-406d-9b80-19b8680ea8ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:57.732400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d874bb-8c20-406d-9b80-19b8680ea8ed service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:57.765817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1173/2308] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:27:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:27:59.082868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8abba-de26-4c85-8e2e-d4fe177ef182 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:59.084242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8abba-de26-4c85-8e2e-d4fe177ef182 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:59.085026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8abba-de26-4c85-8e2e-d4fe177ef182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:59.116681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2fc8abba-de26-4c85-8e2e-d4fe177ef182 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:27:59.119719 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2fc8abba-de26-4c85-8e2e-d4fe177ef182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:27:59.131013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1136/2309] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:27:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:27:59.164967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d874bb-8c20-406d-9b80-19b8680ea8ed service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:27:59.190885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d874bb-8c20-406d-9b80-19b8680ea8ed service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:27:59.201188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b625c47-b809-4b28-8960-c75f86314f1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:27:59.202896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625c47-b809-4b28-8960-c75f86314f1f service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:27:59.203656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625c47-b809-4b28-8960-c75f86314f1f service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:27:59.204334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625c47-b809-4b28-8960-c75f86314f1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:27:59.234027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b625c47-b809-4b28-8960-c75f86314f1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:27:59.237826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b625c47-b809-4b28-8960-c75f86314f1f service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:27:59.278704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1174/2310] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:27:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:28:09.108852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=15 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:09.248514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=40 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:14.495570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8abba-de26-4c85-8e2e-d4fe177ef182 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:14.517845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8abba-de26-4c85-8e2e-d4fe177ef182 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:14.530230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc95749-aa6e-4e9b-9633-7fbc7fe94dda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:14.531338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc95749-aa6e-4e9b-9633-7fbc7fe94dda service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:14.532050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc95749-aa6e-4e9b-9633-7fbc7fe94dda service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:14.532670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc95749-aa6e-4e9b-9633-7fbc7fe94dda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:14.535573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4cc95749-aa6e-4e9b-9633-7fbc7fe94dda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:14.558496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc95749-aa6e-4e9b-9633-7fbc7fe94dda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:14.562067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc95749-aa6e-4e9b-9633-7fbc7fe94dda service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:14.584976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1137/2311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:28:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:28:18.788110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b625c47-b809-4b28-8960-c75f86314f1f service swift] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:18.810828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b625c47-b809-4b28-8960-c75f86314f1f service swift] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:18.838158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a0ace-fbff-42dd-ba1c-51d4a8791ac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:18.839941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a0ace-fbff-42dd-ba1c-51d4a8791ac2 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:18.844283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a0ace-fbff-42dd-ba1c-51d4a8791ac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:18.845614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a0ace-fbff-42dd-ba1c-51d4a8791ac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:18.854075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a32a0ace-fbff-42dd-ba1c-51d4a8791ac2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:18.913628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a0ace-fbff-42dd-ba1c-51d4a8791ac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:18.924187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a0ace-fbff-42dd-ba1c-51d4a8791ac2 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:18.988133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1175/2312] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:28:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:28:20.412443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc95749-aa6e-4e9b-9633-7fbc7fe94dda service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:20.429154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc95749-aa6e-4e9b-9633-7fbc7fe94dda service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:20.439260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdb4fb-d600-4e24-be24-1be93f80d406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:20.440724 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdb4fb-d600-4e24-be24-1be93f80d406 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:20.441223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdb4fb-d600-4e24-be24-1be93f80d406 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:20.441801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdb4fb-d600-4e24-be24-1be93f80d406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:20.444981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-49fdb4fb-d600-4e24-be24-1be93f80d406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:20.468873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdb4fb-d600-4e24-be24-1be93f80d406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:20.471864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdb4fb-d600-4e24-be24-1be93f80d406 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:20.502177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1138/2313] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:28:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:28:25.976588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d7fa3-df4a-4aab-af6e-f044556f0885 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:25.977171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d7fa3-df4a-4aab-af6e-f044556f0885 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:25.977570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d7fa3-df4a-4aab-af6e-f044556f0885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:26.008314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1a0d7fa3-df4a-4aab-af6e-f044556f0885 None None] MFA Rules not processed for user `53505ac8ab6142a6bf0c3d6fe1cc721f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:28:26.009922 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1a0d7fa3-df4a-4aab-af6e-f044556f0885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:26.018453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1176/2314] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:28:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:28:30.483989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=12 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:36.001464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=9 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:36.056632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdb4fb-d600-4e24-be24-1be93f80d406 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:36.071153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdb4fb-d600-4e24-be24-1be93f80d406 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:36.079025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0c5c-6bfd-41e0-a09e-4f9fe7c76417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:36.080226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0c5c-6bfd-41e0-a09e-4f9fe7c76417 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:36.080855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0c5c-6bfd-41e0-a09e-4f9fe7c76417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:36.081375 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0c5c-6bfd-41e0-a09e-4f9fe7c76417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:36.117453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f0c5c-6bfd-41e0-a09e-4f9fe7c76417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:36.123708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f0c5c-6bfd-41e0-a09e-4f9fe7c76417 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:36.153072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1139/2315] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:28:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:28:43.100946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d7fa3-df4a-4aab-af6e-f044556f0885 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:43.119516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d7fa3-df4a-4aab-af6e-f044556f0885 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:43.126964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:43.128118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:43.128618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:43.129132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:43.132625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:43.268619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:43.271508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:43.292330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1177/2316] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:28:46.137120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:48.842095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0c5c-6bfd-41e0-a09e-4f9fe7c76417 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:48.856918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0c5c-6bfd-41e0-a09e-4f9fe7c76417 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:48.865319 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebc57e-5195-44d4-bbcc-dff89fe061d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:48.866463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebc57e-5195-44d4-bbcc-dff89fe061d6 service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:48.866739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebc57e-5195-44d4-bbcc-dff89fe061d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:48.867306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebc57e-5195-44d4-bbcc-dff89fe061d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:48.870313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f5ebc57e-5195-44d4-bbcc-dff89fe061d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:48.896768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebc57e-5195-44d4-bbcc-dff89fe061d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:48.899101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebc57e-5195-44d4-bbcc-dff89fe061d6 service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:48.931516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1140/2317] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:28:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:28:48.954843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:48.957156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:48.981757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3263-5fee-4a24-b67f-dbad63ba2338 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:48.991719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:48.994456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:48.996198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:48.996947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] PATH_INFO: `/v3/users/86f9bcac527c45d6bc81b70e26544924` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:48.999706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f9bcac527c45d6bc81b70e26544924)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:49.002607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:49.061018 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f9bcac527c45d6bc81b70e26544924'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.069382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f9bcac527c45d6bc81b70e26544924'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.070323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f9bcac527c45d6bc81b70e26544924'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.079781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f9bcac527c45d6bc81b70e26544924'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.098328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-752a49f9-6e12-4146-9d06-63a57d138df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f9bcac527c45d6bc81b70e26544924'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.111005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1178/2318] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:28:48 2022] DELETE /identity/v3/users/86f9bcac527c45d6bc81b70e26544924 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:28:49.333237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebc57e-5195-44d4-bbcc-dff89fe061d6 service placement] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:49.352327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebc57e-5195-44d4-bbcc-dff89fe061d6 service placement] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:49.426591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:49.427515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:49.427800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:49.428081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] PATH_INFO: `/v3/projects/5e2e414f37bf4a89ac18e1836c793b0b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:49.430209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e2e414f37bf4a89ac18e1836c793b0b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:49.434714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:49.517324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e2e414f37bf4a89ac18e1836c793b0b', 'request_id': 'req-fd144c8e-0b52-477e-a80f-4d166d8c94c7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.527475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e2e414f37bf4a89ac18e1836c793b0b', 'request_id': 'req-fd144c8e-0b52-477e-a80f-4d166d8c94c7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.535118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e2e414f37bf4a89ac18e1836c793b0b', 'request_id': 'req-fd144c8e-0b52-477e-a80f-4d166d8c94c7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.550619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] The token cache is being invalidate because project 5e2e414f37bf4a89ac18e1836c793b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:28:49.551052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.551880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fd144c8e-0b52-477e-a80f-4d166d8c94c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:49.554176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1141/2319] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:28:49 2022] DELETE /identity/v3/projects/5e2e414f37bf4a89ac18e1836c793b0b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:28:49.560497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ad8a9-5276-4a48-9982-6f8cfe5949a3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:49.560937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ad8a9-5276-4a48-9982-6f8cfe5949a3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:49.561323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ad8a9-5276-4a48-9982-6f8cfe5949a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:49.581478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0f7ad8a9-5276-4a48-9982-6f8cfe5949a3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:28:49.582681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0f7ad8a9-5276-4a48-9982-6f8cfe5949a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:49.657502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1179/2320] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:28:49.669504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460d453-9933-4de7-8393-482286aa9f80 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:49.670049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460d453-9933-4de7-8393-482286aa9f80 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:49.670462 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460d453-9933-4de7-8393-482286aa9f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:49.695162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b460d453-9933-4de7-8393-482286aa9f80 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:28:49.696394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b460d453-9933-4de7-8393-482286aa9f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:49.705975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1142/2321] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:28:49.710668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ad8a9-5276-4a48-9982-6f8cfe5949a3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:49.722593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ad8a9-5276-4a48-9982-6f8cfe5949a3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:49.727511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0284290-74b3-4b97-9ad6-31c659c44ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:49.728195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0284290-74b3-4b97-9ad6-31c659c44ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:49.728473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0284290-74b3-4b97-9ad6-31c659c44ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:49.728767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0284290-74b3-4b97-9ad6-31c659c44ce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:49.729496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0284290-74b3-4b97-9ad6-31c659c44ce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:28:49.730199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0284290-74b3-4b97-9ad6-31c659c44ce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:49.732372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0284290-74b3-4b97-9ad6-31c659c44ce6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:49.734417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:49.751352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1180/2322] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:28:49.755638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460d453-9933-4de7-8393-482286aa9f80 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:49.766886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460d453-9933-4de7-8393-482286aa9f80 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:49.771790 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d538ab5-66bb-4f26-a5e8-0f6ea4816768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:49.772465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d538ab5-66bb-4f26-a5e8-0f6ea4816768 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:49.772752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d538ab5-66bb-4f26-a5e8-0f6ea4816768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:49.773022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d538ab5-66bb-4f26-a5e8-0f6ea4816768 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:49.773734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d538ab5-66bb-4f26-a5e8-0f6ea4816768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:49.775686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d538ab5-66bb-4f26-a5e8-0f6ea4816768 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:49.820022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1143/2323] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:28:49 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:28:49.825731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0284290-74b3-4b97-9ad6-31c659c44ce6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:49.837460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0284290-74b3-4b97-9ad6-31c659c44ce6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:49.915768 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d9d9f-1ecd-4d73-baef-5171103e9a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:49.916464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d9d9f-1ecd-4d73-baef-5171103e9a2d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:49.916806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d9d9f-1ecd-4d73-baef-5171103e9a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:49.919343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d9d9f-1ecd-4d73-baef-5171103e9a2d admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:49.919343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d9d9f-1ecd-4d73-baef-5171103e9a2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:49.920513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d9d9f-1ecd-4d73-baef-5171103e9a2d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:49.950092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1181/2324] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:28:49 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:28:49.956045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d538ab5-66bb-4f26-a5e8-0f6ea4816768 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:49.969692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d538ab5-66bb-4f26-a5e8-0f6ea4816768 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:49.975540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5558e6-528b-43fc-b4cb-a94bf07b2efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:49.976494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5558e6-528b-43fc-b4cb-a94bf07b2efd admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:49.976953 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5558e6-528b-43fc-b4cb-a94bf07b2efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:49.977395 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5558e6-528b-43fc-b4cb-a94bf07b2efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:49.978370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5558e6-528b-43fc-b4cb-a94bf07b2efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:49.980687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5558e6-528b-43fc-b4cb-a94bf07b2efd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:49.992565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1144/2325] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:28:49.997855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d9d9f-1ecd-4d73-baef-5171103e9a2d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:50.017641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d9d9f-1ecd-4d73-baef-5171103e9a2d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:50.026563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a87cb-5687-453d-8ecb-c9f9513d809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:50.028554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a87cb-5687-453d-8ecb-c9f9513d809f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:50.029130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a87cb-5687-453d-8ecb-c9f9513d809f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:50.029643 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a87cb-5687-453d-8ecb-c9f9513d809f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:50.031210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a87cb-5687-453d-8ecb-c9f9513d809f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:50.034323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a87cb-5687-453d-8ecb-c9f9513d809f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:50.055078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1182/2326] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:28:50.060239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5558e6-528b-43fc-b4cb-a94bf07b2efd admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:50.080709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5558e6-528b-43fc-b4cb-a94bf07b2efd admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:50.089789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378976fc-4330-4f56-a49d-3ee3ec478d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:50.092345 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378976fc-4330-4f56-a49d-3ee3ec478d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:50.093105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378976fc-4330-4f56-a49d-3ee3ec478d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:50.093607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378976fc-4330-4f56-a49d-3ee3ec478d69 admin admin] PATH_INFO: `/v3/projects/594246afbcdc478d81b85c5b525ae1e9/users/2aeeaf8afea14478b96d2e8e4c77b9d7/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:50.126087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378976fc-4330-4f56-a49d-3ee3ec478d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594246afbcdc478d81b85c5b525ae1e9, user_id=2aeeaf8afea14478b96d2e8e4c77b9d7, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:50.136757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378976fc-4330-4f56-a49d-3ee3ec478d69 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:50.162769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1145/2327] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:28:50 2022] PUT /identity/v3/projects/594246afbcdc478d81b85c5b525ae1e9/users/2aeeaf8afea14478b96d2e8e4c77b9d7/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:28:50.176973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d447c-af85-440a-88fa-75aeb25159ff None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:50.177253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d447c-af85-440a-88fa-75aeb25159ff None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:50.177854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d447c-af85-440a-88fa-75aeb25159ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:50.223146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-dd4d447c-af85-440a-88fa-75aeb25159ff None None] MFA Rules not processed for user `2aeeaf8afea14478b96d2e8e4c77b9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:28:50.224551 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dd4d447c-af85-440a-88fa-75aeb25159ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:50.304142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1183/2328] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:28:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:28:50.317607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378976fc-4330-4f56-a49d-3ee3ec478d69 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:50.320046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-378976fc-4330-4f56-a49d-3ee3ec478d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:50.329768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:50.412835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378976fc-4330-4f56-a49d-3ee3ec478d69 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:50.419300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a8ce3-2a91-4f93-9666-c8549346109c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:50.419994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a8ce3-2a91-4f93-9666-c8549346109c service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:50.420258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a8ce3-2a91-4f93-9666-c8549346109c service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:50.420541 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a8ce3-2a91-4f93-9666-c8549346109c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:50.430401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a8ce3-2a91-4f93-9666-c8549346109c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:50.432098 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a8ce3-2a91-4f93-9666-c8549346109c service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:50.443471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1146/2329] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:28:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:28:58.192265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d447c-af85-440a-88fa-75aeb25159ff None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:58.298845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d447c-af85-440a-88fa-75aeb25159ff None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:58.307234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:58.311130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:58.311615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:58.312094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] PATH_INFO: `/v3/users/2aeeaf8afea14478b96d2e8e4c77b9d7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:58.314698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aeeaf8afea14478b96d2e8e4c77b9d7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:58.318383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:58.322925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:58.398866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aeeaf8afea14478b96d2e8e4c77b9d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:58.410469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aeeaf8afea14478b96d2e8e4c77b9d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:58.411425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aeeaf8afea14478b96d2e8e4c77b9d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:58.428692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aeeaf8afea14478b96d2e8e4c77b9d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:58.448187 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ed450c9c-bd4b-4828-843d-6c844d04a3e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aeeaf8afea14478b96d2e8e4c77b9d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:58.458332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1184/2330] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:28:58 2022] DELETE /identity/v3/users/2aeeaf8afea14478b96d2e8e4c77b9d7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:28:58.463409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23664973-f305-490d-926b-653e94b1b6c2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:58.465513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23664973-f305-490d-926b-653e94b1b6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:58.466085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23664973-f305-490d-926b-653e94b1b6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:58.494483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-23664973-f305-490d-926b-653e94b1b6c2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:28:58.496952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-23664973-f305-490d-926b-653e94b1b6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:58.528695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405002cb-bbca-46a5-bb3d-4a4c88149aa5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:58.529822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405002cb-bbca-46a5-bb3d-4a4c88149aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:58.530387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405002cb-bbca-46a5-bb3d-4a4c88149aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:58.567443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-405002cb-bbca-46a5-bb3d-4a4c88149aa5 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:28:58.569121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-405002cb-bbca-46a5-bb3d-4a4c88149aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:58.642041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1147/2331] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:28:58.657220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1185/2332] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:28:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:28:58.663829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23664973-f305-490d-926b-653e94b1b6c2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:58.669047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-23664973-f305-490d-926b-653e94b1b6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:58.703333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405002cb-bbca-46a5-bb3d-4a4c88149aa5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:58.706396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-405002cb-bbca-46a5-bb3d-4a4c88149aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:58.798839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23664973-f305-490d-926b-653e94b1b6c2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:58.805599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64960223-ab5a-4391-8834-df1663c54390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:58.806680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64960223-ab5a-4391-8834-df1663c54390 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:58.807163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64960223-ab5a-4391-8834-df1663c54390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:58.808802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64960223-ab5a-4391-8834-df1663c54390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:58.823969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64960223-ab5a-4391-8834-df1663c54390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:58.826690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64960223-ab5a-4391-8834-df1663c54390 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:58.850536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1148/2333] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:28:58.892552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405002cb-bbca-46a5-bb3d-4a4c88149aa5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:58.898944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9fa58-e2f2-44da-b65f-bbad9407067e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:58.899650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9fa58-e2f2-44da-b65f-bbad9407067e service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:58.900130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9fa58-e2f2-44da-b65f-bbad9407067e service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:58.900444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9fa58-e2f2-44da-b65f-bbad9407067e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:58.914094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9fa58-e2f2-44da-b65f-bbad9407067e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:58.917892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9fa58-e2f2-44da-b65f-bbad9407067e service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:58.936232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1186/2334] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:28:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 14:28:59.386855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64960223-ab5a-4391-8834-df1663c54390 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:59.390379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-64960223-ab5a-4391-8834-df1663c54390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:59.424441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64960223-ab5a-4391-8834-df1663c54390 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:28:59.437029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:28:59.440328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:59.441310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:59.442138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] PATH_INFO: `/v3/projects/594246afbcdc478d81b85c5b525ae1e9` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:59.444976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] RBAC: Authorizing `identity:delete_project(project_id=594246afbcdc478d81b85c5b525ae1e9)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:28:59.450062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:28:59.530246 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:59.572005 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594246afbcdc478d81b85c5b525ae1e9', 'request_id': 'req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:59.587382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594246afbcdc478d81b85c5b525ae1e9', 'request_id': 'req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:59.603893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594246afbcdc478d81b85c5b525ae1e9', 'request_id': 'req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:59.631652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] The token cache is being invalidate because project 594246afbcdc478d81b85c5b525ae1e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:28:59.632025 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:59.636603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-06cd9bf0-7ccd-4e42-969e-d45723f2c91d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:28:59.641815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1149/2335] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:28:59 2022] DELETE /identity/v3/projects/594246afbcdc478d81b85c5b525ae1e9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:28:59.645555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28523b4a-3b9b-466f-af0d-aa8a79d5a750 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:59.646385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28523b4a-3b9b-466f-af0d-aa8a79d5a750 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:59.646728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28523b4a-3b9b-466f-af0d-aa8a79d5a750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:59.682255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-28523b4a-3b9b-466f-af0d-aa8a79d5a750 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:28:59.684455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-28523b4a-3b9b-466f-af0d-aa8a79d5a750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:59.705013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:28:59.832202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1187/2336] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:28:59.845578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d44b5-e794-48f6-8d83-f7e1f69561f4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:28:59.846155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d44b5-e794-48f6-8d83-f7e1f69561f4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:28:59.846662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d44b5-e794-48f6-8d83-f7e1f69561f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:28:59.916567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-648d44b5-e794-48f6-8d83-f7e1f69561f4 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:28:59.916567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-648d44b5-e794-48f6-8d83-f7e1f69561f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:28:59.977544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1150/2337] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:28:59.978020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28523b4a-3b9b-466f-af0d-aa8a79d5a750 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:28:59.996652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28523b4a-3b9b-466f-af0d-aa8a79d5a750 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:00.002030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b4382-f1f2-450e-984e-8354f77b64c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:00.002734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4382-f1f2-450e-984e-8354f77b64c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:00.003026 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4382-f1f2-450e-984e-8354f77b64c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:00.003300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4382-f1f2-450e-984e-8354f77b64c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:00.004231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b4382-f1f2-450e-984e-8354f77b64c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:29:00.004550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b4382-f1f2-450e-984e-8354f77b64c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:00.006287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b4382-f1f2-450e-984e-8354f77b64c4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:00.029573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1188/2338] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:29:00.036257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d44b5-e794-48f6-8d83-f7e1f69561f4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:00.066464 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d44b5-e794-48f6-8d83-f7e1f69561f4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:00.076048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37040e45-9b4f-4d67-81f2-39e536f06351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:00.076739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37040e45-9b4f-4d67-81f2-39e536f06351 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:00.077028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37040e45-9b4f-4d67-81f2-39e536f06351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:00.077315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37040e45-9b4f-4d67-81f2-39e536f06351 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:00.078086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37040e45-9b4f-4d67-81f2-39e536f06351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:00.080687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37040e45-9b4f-4d67-81f2-39e536f06351 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:00.172794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1151/2339] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:00 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:00.178155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b4382-f1f2-450e-984e-8354f77b64c4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:00.197784 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b4382-f1f2-450e-984e-8354f77b64c4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:00.326794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a841eb0-2a51-496a-b3db-11f5f96c23f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:00.327677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a841eb0-2a51-496a-b3db-11f5f96c23f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:00.327977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a841eb0-2a51-496a-b3db-11f5f96c23f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:00.328372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a841eb0-2a51-496a-b3db-11f5f96c23f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:00.329764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a841eb0-2a51-496a-b3db-11f5f96c23f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:00.332111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a841eb0-2a51-496a-b3db-11f5f96c23f5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:00.360181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1189/2340] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:00 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:29:00.364905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37040e45-9b4f-4d67-81f2-39e536f06351 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:00.390353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37040e45-9b4f-4d67-81f2-39e536f06351 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:00.399007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ed6c0-90d3-426e-b2b2-d086c6c6ac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:00.402266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ed6c0-90d3-426e-b2b2-d086c6c6ac58 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:00.404900 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ed6c0-90d3-426e-b2b2-d086c6c6ac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:00.405886 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ed6c0-90d3-426e-b2b2-d086c6c6ac58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:00.409637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ed6c0-90d3-426e-b2b2-d086c6c6ac58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:00.412751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ed6c0-90d3-426e-b2b2-d086c6c6ac58 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:00.437246 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1152/2341] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:29:00.442587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a841eb0-2a51-496a-b3db-11f5f96c23f5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:00.461013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a841eb0-2a51-496a-b3db-11f5f96c23f5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:00.469649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232db5d4-b2d5-45aa-9321-4763b205394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:00.471692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232db5d4-b2d5-45aa-9321-4763b205394b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:00.471692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232db5d4-b2d5-45aa-9321-4763b205394b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:00.482801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232db5d4-b2d5-45aa-9321-4763b205394b admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:00.484242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232db5d4-b2d5-45aa-9321-4763b205394b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:00.487316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232db5d4-b2d5-45aa-9321-4763b205394b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:00.522578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ed6c0-90d3-426e-b2b2-d086c6c6ac58 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:00.531487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1190/2342] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:00.546934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ed6c0-90d3-426e-b2b2-d086c6c6ac58 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:00.559340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6ef2-600d-48e5-ad8a-6f071031c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:00.560903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6ef2-600d-48e5-ad8a-6f071031c0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:00.561392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6ef2-600d-48e5-ad8a-6f071031c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:00.561899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6ef2-600d-48e5-ad8a-6f071031c0d2 admin admin] PATH_INFO: `/v3/projects/f2abbae84ab74993b3b9910197746ce3/users/088524c3a77a4c3d858e0b60117a0ada/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:00.584870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e6ef2-600d-48e5-ad8a-6f071031c0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2abbae84ab74993b3b9910197746ce3, user_id=088524c3a77a4c3d858e0b60117a0ada, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:00.589071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e6ef2-600d-48e5-ad8a-6f071031c0d2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:00.608416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1153/2343] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:00 2022] PUT /identity/v3/projects/f2abbae84ab74993b3b9910197746ce3/users/088524c3a77a4c3d858e0b60117a0ada/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:00.623336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8aead7-ccca-45ea-8341-a219a0727b8a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:00.623700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8aead7-ccca-45ea-8341-a219a0727b8a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:00.623979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8aead7-ccca-45ea-8341-a219a0727b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:00.657773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-fc8aead7-ccca-45ea-8341-a219a0727b8a None None] MFA Rules not processed for user `088524c3a77a4c3d858e0b60117a0ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:00.659033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fc8aead7-ccca-45ea-8341-a219a0727b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:00.839659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1191/2344] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:00.859374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6ef2-600d-48e5-ad8a-6f071031c0d2 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:00.865296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-051e6ef2-600d-48e5-ad8a-6f071031c0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:01.060588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6ef2-600d-48e5-ad8a-6f071031c0d2 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:01.071347 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6914c2-eb7a-47be-bf8a-772873794b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:01.073340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6914c2-eb7a-47be-bf8a-772873794b6f service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:01.074152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6914c2-eb7a-47be-bf8a-772873794b6f service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:01.074889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6914c2-eb7a-47be-bf8a-772873794b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:01.091770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6914c2-eb7a-47be-bf8a-772873794b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:01.094695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6914c2-eb7a-47be-bf8a-772873794b6f service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:01.113771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1154/2345] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:01.555959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8aead7-ccca-45ea-8341-a219a0727b8a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:01.591065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:01.741759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8aead7-ccca-45ea-8341-a219a0727b8a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:01.755969 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:01.758254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:01.759155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:01.759887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] PATH_INFO: `/v3/users/088524c3a77a4c3d858e0b60117a0ada` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:01.763320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] RBAC: Authorizing `identity:delete_user(user_id=088524c3a77a4c3d858e0b60117a0ada)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:01.767092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:01.881455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088524c3a77a4c3d858e0b60117a0ada'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:01.894728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088524c3a77a4c3d858e0b60117a0ada'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:01.895928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088524c3a77a4c3d858e0b60117a0ada'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:01.909913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088524c3a77a4c3d858e0b60117a0ada'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:01.951588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088524c3a77a4c3d858e0b60117a0ada'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:01.964511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1192/2346] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:01 2022] DELETE /identity/v3/users/088524c3a77a4c3d858e0b60117a0ada => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:01.968999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892d0c6-e152-4631-a0b3-b7a2d4da4d9e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:01.970711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892d0c6-e152-4631-a0b3-b7a2d4da4d9e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:01.971208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892d0c6-e152-4631-a0b3-b7a2d4da4d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:02.005032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3892d0c6-e152-4631-a0b3-b7a2d4da4d9e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:02.007636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3892d0c6-e152-4631-a0b3-b7a2d4da4d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:02.091619 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:02.157069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1155/2347] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:02.175865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:02.177754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:02.312743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd6e17-220c-44e1-9e8b-dd5b7eb5538f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:02.325921 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d01b1-ec9f-4701-a91a-c0ca8fb4e471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:02.327955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d01b1-ec9f-4701-a91a-c0ca8fb4e471 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:02.328471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d01b1-ec9f-4701-a91a-c0ca8fb4e471 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:02.328986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d01b1-ec9f-4701-a91a-c0ca8fb4e471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:02.357890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d01b1-ec9f-4701-a91a-c0ca8fb4e471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:02.360675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d01b1-ec9f-4701-a91a-c0ca8fb4e471 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:02.385241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1193/2348] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:29:02.884580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892d0c6-e152-4631-a0b3-b7a2d4da4d9e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:02.889294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3892d0c6-e152-4631-a0b3-b7a2d4da4d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:02.939017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892d0c6-e152-4631-a0b3-b7a2d4da4d9e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:02.966601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:02.968407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:02.968806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:02.969194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] PATH_INFO: `/v3/projects/f2abbae84ab74993b3b9910197746ce3` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:02.994439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2abbae84ab74993b3b9910197746ce3)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:02.998500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:03.105224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2abbae84ab74993b3b9910197746ce3', 'request_id': 'req-0a309c8e-c32d-413e-958f-1bedf7fd15f6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:03.125084 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2abbae84ab74993b3b9910197746ce3', 'request_id': 'req-0a309c8e-c32d-413e-958f-1bedf7fd15f6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:03.137396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2abbae84ab74993b3b9910197746ce3', 'request_id': 'req-0a309c8e-c32d-413e-958f-1bedf7fd15f6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:03.161919 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] The token cache is being invalidate because project f2abbae84ab74993b3b9910197746ce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:03.162714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:03.165454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a309c8e-c32d-413e-958f-1bedf7fd15f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:03.175740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1156/2349] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:02 2022] DELETE /identity/v3/projects/f2abbae84ab74993b3b9910197746ce3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:03.178679 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2b58b-f0cf-4bc6-a32b-d0ef9402843c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:03.179544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2b58b-f0cf-4bc6-a32b-d0ef9402843c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:03.179859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2b58b-f0cf-4bc6-a32b-d0ef9402843c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:03.201658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-67b2b58b-f0cf-4bc6-a32b-d0ef9402843c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:03.202955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-67b2b58b-f0cf-4bc6-a32b-d0ef9402843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:03.241603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:03.297558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1194/2350] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:03.306175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952602db-9751-4830-9c83-d16d57dec275 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:03.306446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952602db-9751-4830-9c83-d16d57dec275 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:03.306738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952602db-9751-4830-9c83-d16d57dec275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:03.329011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-952602db-9751-4830-9c83-d16d57dec275 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:03.330310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-952602db-9751-4830-9c83-d16d57dec275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:03.338814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1157/2351] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:03.344578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2b58b-f0cf-4bc6-a32b-d0ef9402843c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:03.360578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2b58b-f0cf-4bc6-a32b-d0ef9402843c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:03.367318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa5fc-7e7d-4447-9816-2ed05216e356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:03.368044 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa5fc-7e7d-4447-9816-2ed05216e356 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:03.368418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa5fc-7e7d-4447-9816-2ed05216e356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:03.368700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa5fc-7e7d-4447-9816-2ed05216e356 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:03.369502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faa5fc-7e7d-4447-9816-2ed05216e356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:29:03.370915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faa5fc-7e7d-4447-9816-2ed05216e356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:03.372736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faa5fc-7e7d-4447-9816-2ed05216e356 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:03.396646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1195/2352] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:29:03.400917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952602db-9751-4830-9c83-d16d57dec275 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:03.420146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952602db-9751-4830-9c83-d16d57dec275 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:03.428738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42c4a6-1f46-445a-8b82-bb2a5954d52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:03.429906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42c4a6-1f46-445a-8b82-bb2a5954d52a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:03.430373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42c4a6-1f46-445a-8b82-bb2a5954d52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:03.430847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42c4a6-1f46-445a-8b82-bb2a5954d52a admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:03.431757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42c4a6-1f46-445a-8b82-bb2a5954d52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:03.435351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42c4a6-1f46-445a-8b82-bb2a5954d52a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:03.500973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa5fc-7e7d-4447-9816-2ed05216e356 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:03.502248 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1158/2353] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:03 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:03.529620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa5fc-7e7d-4447-9816-2ed05216e356 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:03.663571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b00f5-b68d-4c98-b3c8-4e6420e7cefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:03.665014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b00f5-b68d-4c98-b3c8-4e6420e7cefc admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:03.665758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b00f5-b68d-4c98-b3c8-4e6420e7cefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:03.666391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b00f5-b68d-4c98-b3c8-4e6420e7cefc admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:03.668156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b00f5-b68d-4c98-b3c8-4e6420e7cefc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:03.671537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b00f5-b68d-4c98-b3c8-4e6420e7cefc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:03.708029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1196/2354] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:03 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:29:03.712770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42c4a6-1f46-445a-8b82-bb2a5954d52a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:03.732822 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42c4a6-1f46-445a-8b82-bb2a5954d52a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:03.740838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e182780-1c06-4e4b-bc4a-9338805962e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:03.742063 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e182780-1c06-4e4b-bc4a-9338805962e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:03.742629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e182780-1c06-4e4b-bc4a-9338805962e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:03.743194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e182780-1c06-4e4b-bc4a-9338805962e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:03.744633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e182780-1c06-4e4b-bc4a-9338805962e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:03.749067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e182780-1c06-4e4b-bc4a-9338805962e0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:03.773486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1159/2355] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:03.774216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b00f5-b68d-4c98-b3c8-4e6420e7cefc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:03.796172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b00f5-b68d-4c98-b3c8-4e6420e7cefc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:03.804328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde12a4-3c26-4da7-a1dd-ef0a57a569d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:03.805242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde12a4-3c26-4da7-a1dd-ef0a57a569d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:03.805750 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde12a4-3c26-4da7-a1dd-ef0a57a569d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:03.806243 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde12a4-3c26-4da7-a1dd-ef0a57a569d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:03.807396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde12a4-3c26-4da7-a1dd-ef0a57a569d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:03.809358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde12a4-3c26-4da7-a1dd-ef0a57a569d2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:03.830090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1197/2356] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:29:03.832090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e182780-1c06-4e4b-bc4a-9338805962e0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:03.849813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e182780-1c06-4e4b-bc4a-9338805962e0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:03.859713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570a14-457b-4513-9d9a-fb1a0cb681c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:03.862399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570a14-457b-4513-9d9a-fb1a0cb681c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:03.862842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570a14-457b-4513-9d9a-fb1a0cb681c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:03.863340 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570a14-457b-4513-9d9a-fb1a0cb681c0 admin admin] PATH_INFO: `/v3/projects/9b145c2c6c5447679b3cd5c3a3cd7d9e/users/02247645e81441c7a8a501d34227d1bf/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:03.897543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8570a14-457b-4513-9d9a-fb1a0cb681c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b145c2c6c5447679b3cd5c3a3cd7d9e, user_id=02247645e81441c7a8a501d34227d1bf, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:03.900692 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8570a14-457b-4513-9d9a-fb1a0cb681c0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:03.927015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1160/2357] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:03 2022] PUT /identity/v3/projects/9b145c2c6c5447679b3cd5c3a3cd7d9e/users/02247645e81441c7a8a501d34227d1bf/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:03.940610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce566bb7-8986-4917-9de0-85bb76854a4f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:03.941060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce566bb7-8986-4917-9de0-85bb76854a4f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:03.941498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce566bb7-8986-4917-9de0-85bb76854a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:03.988844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ce566bb7-8986-4917-9de0-85bb76854a4f None None] MFA Rules not processed for user `02247645e81441c7a8a501d34227d1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:03.990962 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ce566bb7-8986-4917-9de0-85bb76854a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:04.145939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1198/2358] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:04.150347 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570a14-457b-4513-9d9a-fb1a0cb681c0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:04.320366 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:04.327425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570a14-457b-4513-9d9a-fb1a0cb681c0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:04.341975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2db76b-3e99-4653-85a9-8b9ca1be04cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:04.343359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2db76b-3e99-4653-85a9-8b9ca1be04cc admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:04.343834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2db76b-3e99-4653-85a9-8b9ca1be04cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:04.344358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2db76b-3e99-4653-85a9-8b9ca1be04cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:04.345163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2db76b-3e99-4653-85a9-8b9ca1be04cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:04.348230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2db76b-3e99-4653-85a9-8b9ca1be04cc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:04.396708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1161/2359] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:04 2022] POST /identity/v3/projects => generated 379 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:04.399681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce566bb7-8986-4917-9de0-85bb76854a4f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:04.412622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce566bb7-8986-4917-9de0-85bb76854a4f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:04.460105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:04.521553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2da17d-034d-4fef-ba3d-dd0ad0453e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:04.523310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2da17d-034d-4fef-ba3d-dd0ad0453e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:04.524073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2da17d-034d-4fef-ba3d-dd0ad0453e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:04.524511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2da17d-034d-4fef-ba3d-dd0ad0453e99 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:04.525863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2da17d-034d-4fef-ba3d-dd0ad0453e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:04.534443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2da17d-034d-4fef-ba3d-dd0ad0453e99 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:04.593429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1199/2360] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:04 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:29:04.595550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2db76b-3e99-4653-85a9-8b9ca1be04cc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:04.617780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2db76b-3e99-4653-85a9-8b9ca1be04cc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:04.629756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f1772-1674-48b3-b230-b46a6bf56e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:04.630878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f1772-1674-48b3-b230-b46a6bf56e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:04.631344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f1772-1674-48b3-b230-b46a6bf56e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:04.631804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f1772-1674-48b3-b230-b46a6bf56e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:04.635705 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f1772-1674-48b3-b230-b46a6bf56e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:04.640762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f1772-1674-48b3-b230-b46a6bf56e07 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:04.664538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1162/2361] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:29:04.677779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2da17d-034d-4fef-ba3d-dd0ad0453e99 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:04.705642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2da17d-034d-4fef-ba3d-dd0ad0453e99 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:04.721516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafad54-07f2-4751-a34b-d0959edec197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:04.724120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafad54-07f2-4751-a34b-d0959edec197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:04.724678 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafad54-07f2-4751-a34b-d0959edec197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:04.725219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafad54-07f2-4751-a34b-d0959edec197 admin admin] PATH_INFO: `/v3/projects/67df373e88c949f2865c164a49eb4cca/users/ec400ab6c140471896af62d2f37f42bc/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:04.755234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafad54-07f2-4751-a34b-d0959edec197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67df373e88c949f2865c164a49eb4cca, user_id=ec400ab6c140471896af62d2f37f42bc, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:04.762341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafad54-07f2-4751-a34b-d0959edec197 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:04.795836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1200/2362] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:04 2022] PUT /identity/v3/projects/67df373e88c949f2865c164a49eb4cca/users/ec400ab6c140471896af62d2f37f42bc/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:04.812227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f1772-1674-48b3-b230-b46a6bf56e07 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:04.832042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f1772-1674-48b3-b230-b46a6bf56e07 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:04.987293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea73834-d726-48b2-85c9-b68fac8a1020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:04.988500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea73834-d726-48b2-85c9-b68fac8a1020 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:04.989082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea73834-d726-48b2-85c9-b68fac8a1020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:04.989527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea73834-d726-48b2-85c9-b68fac8a1020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:04.990908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea73834-d726-48b2-85c9-b68fac8a1020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:04.993998 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea73834-d726-48b2-85c9-b68fac8a1020 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:05.039236 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1163/2363] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:29:05.042923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafad54-07f2-4751-a34b-d0959edec197 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:05.072319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafad54-07f2-4751-a34b-d0959edec197 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:05.083460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99047c86-0a8d-4d4d-9561-14a96c4f6373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:05.087274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99047c86-0a8d-4d4d-9561-14a96c4f6373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:05.088369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99047c86-0a8d-4d4d-9561-14a96c4f6373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:05.091148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99047c86-0a8d-4d4d-9561-14a96c4f6373 admin admin] PATH_INFO: `/v3/projects/67df373e88c949f2865c164a49eb4cca/users/ec400ab6c140471896af62d2f37f42bc/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:05.095999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99047c86-0a8d-4d4d-9561-14a96c4f6373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67df373e88c949f2865c164a49eb4cca, user_id=ec400ab6c140471896af62d2f37f42bc, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:05.102369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99047c86-0a8d-4d4d-9561-14a96c4f6373 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:05.126079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1201/2364] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:05 2022] PUT /identity/v3/projects/67df373e88c949f2865c164a49eb4cca/users/ec400ab6c140471896af62d2f37f42bc/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:05.149530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1166f-d95c-4975-a6e3-696cb13f1d87 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:05.151196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1166f-d95c-4975-a6e3-696cb13f1d87 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:05.154279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1166f-d95c-4975-a6e3-696cb13f1d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:05.205258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-72e1166f-d95c-4975-a6e3-696cb13f1d87 None None] MFA Rules not processed for user `ec400ab6c140471896af62d2f37f42bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:05.207923 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-72e1166f-d95c-4975-a6e3-696cb13f1d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:05.378172 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1164/2365] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:05.404774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99047c86-0a8d-4d4d-9561-14a96c4f6373 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:05.408512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-99047c86-0a8d-4d4d-9561-14a96c4f6373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:05.609418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99047c86-0a8d-4d4d-9561-14a96c4f6373 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:05.613998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9db07-6946-49e8-b787-93b1f0a293a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:05.619069 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9db07-6946-49e8-b787-93b1f0a293a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:05.619595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9db07-6946-49e8-b787-93b1f0a293a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:05.620868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9db07-6946-49e8-b787-93b1f0a293a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:05.632692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9db07-6946-49e8-b787-93b1f0a293a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:05.634292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9db07-6946-49e8-b787-93b1f0a293a7 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:05.654300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1202/2366] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:29:06.368032 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1166f-d95c-4975-a6e3-696cb13f1d87 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:06.386936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1166f-d95c-4975-a6e3-696cb13f1d87 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:06.396185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489e79d-ba77-4bbb-aa3f-262febc767c9 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec400ab6c140471896af62d2f37f42bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67df373e88c949f2865c164a49eb4cca', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:06.397132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489e79d-ba77-4bbb-aa3f-262febc767c9 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:06.397637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489e79d-ba77-4bbb-aa3f-262febc767c9 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:06.398236 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489e79d-ba77-4bbb-aa3f-262febc767c9 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:06.399678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489e79d-ba77-4bbb-aa3f-262febc767c9 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:29:06.400456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489e79d-ba77-4bbb-aa3f-262febc767c9 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:06.418008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489e79d-ba77-4bbb-aa3f-262febc767c9 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:06.448603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1165/2367] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:29:06.455278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9db07-6946-49e8-b787-93b1f0a293a7 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:06.466444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9db07-6946-49e8-b787-93b1f0a293a7 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:06.498860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ead71-c763-4554-88aa-b28795658c4a tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec400ab6c140471896af62d2f37f42bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67df373e88c949f2865c164a49eb4cca', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:06.499560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ead71-c763-4554-88aa-b28795658c4a tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:06.499973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ead71-c763-4554-88aa-b28795658c4a tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:06.500412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ead71-c763-4554-88aa-b28795658c4a tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:06.501246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ead71-c763-4554-88aa-b28795658c4a tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:06.503410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ead71-c763-4554-88aa-b28795658c4a tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:06.562760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1203/2368] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:06 2022] POST /identity/v3/projects => generated 355 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:06.569767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489e79d-ba77-4bbb-aa3f-262febc767c9 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:06.593849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489e79d-ba77-4bbb-aa3f-262febc767c9 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:06.672145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:06.719239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec400ab6c140471896af62d2f37f42bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67df373e88c949f2865c164a49eb4cca', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:06.719928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:06.720220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:06.720500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:06.721451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:29:06.721799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:06.723576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:06.748069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1166/2369] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:29:06.799545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ead71-c763-4554-88aa-b28795658c4a tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:06.819881 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ead71-c763-4554-88aa-b28795658c4a tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:06.833208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec400ab6c140471896af62d2f37f42bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67df373e88c949f2865c164a49eb4cca', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:06.835759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:06.836491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:06.837063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] PATH_INFO: `/v3/projects/8c05dbf002884da6acff235fdd1bf534` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:06.840509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8c05dbf002884da6acff235fdd1bf534)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:06.844440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:06.958695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c05dbf002884da6acff235fdd1bf534', 'request_id': 'req-23a24359-68b4-4c21-bc07-7eef5778fa1e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:06.983003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c05dbf002884da6acff235fdd1bf534', 'request_id': 'req-23a24359-68b4-4c21-bc07-7eef5778fa1e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:06.985865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:INSERT=7,keystone:DELETE=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:07.018150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c05dbf002884da6acff235fdd1bf534', 'request_id': 'req-23a24359-68b4-4c21-bc07-7eef5778fa1e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:07.057112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] The token cache is being invalidate because project 8c05dbf002884da6acff235fdd1bf534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:07.059189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:07.060729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-23a24359-68b4-4c21-bc07-7eef5778fa1e tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:07.076168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1204/2370] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:06 2022] DELETE /identity/v3/projects/8c05dbf002884da6acff235fdd1bf534 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:07.471566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:07.474412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:07.656803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f42b6-5688-4615-9ff1-c94b7b9e8778 tempest-VolumeQuotaClassesTest-414896322 tempest-VolumeQuotaClassesTest-414896322-project-admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:07.805929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:07.807042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:07.808047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:07.808689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] PATH_INFO: `/v3/users/02247645e81441c7a8a501d34227d1bf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:07.810797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=02247645e81441c7a8a501d34227d1bf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:07.816988 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:07.917586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02247645e81441c7a8a501d34227d1bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:07.935679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02247645e81441c7a8a501d34227d1bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:07.939081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02247645e81441c7a8a501d34227d1bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:07.987908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02247645e81441c7a8a501d34227d1bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:07.990030 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02247645e81441c7a8a501d34227d1bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:08.013285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:08.024639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1167/2371] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:07 2022] DELETE /identity/v3/users/02247645e81441c7a8a501d34227d1bf => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:08.032617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1bfe8-71dc-49c2-a731-3b549c5cdb13 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:08.034091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1bfe8-71dc-49c2-a731-3b549c5cdb13 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:08.034091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1bfe8-71dc-49c2-a731-3b549c5cdb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:08.076340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6bf1bfe8-71dc-49c2-a731-3b549c5cdb13 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:08.078588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6bf1bfe8-71dc-49c2-a731-3b549c5cdb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:08.263557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1205/2372] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:08.270545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:08.272497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:08.477595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a7992-2022-4128-8cf6-bbd88e9b6c1e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:08.487108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8be02d-e653-4522-93d7-098f71917d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:08.487941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8be02d-e653-4522-93d7-098f71917d35 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:08.488354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8be02d-e653-4522-93d7-098f71917d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:08.488703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8be02d-e653-4522-93d7-098f71917d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:08.514571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8be02d-e653-4522-93d7-098f71917d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:08.516318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8be02d-e653-4522-93d7-098f71917d35 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:08.552924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1168/2373] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:09.021393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1bfe8-71dc-49c2-a731-3b549c5cdb13 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:09.025863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6bf1bfe8-71dc-49c2-a731-3b549c5cdb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:09.078968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1bfe8-71dc-49c2-a731-3b549c5cdb13 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:09.097387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:09.099443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:09.100219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:09.100946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] PATH_INFO: `/v3/projects/9b145c2c6c5447679b3cd5c3a3cd7d9e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:09.105338 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b145c2c6c5447679b3cd5c3a3cd7d9e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:09.109677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:09.248729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b145c2c6c5447679b3cd5c3a3cd7d9e', 'request_id': 'req-2cce1740-2aad-4df6-8002-1b47a7cd47db'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.273926 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b145c2c6c5447679b3cd5c3a3cd7d9e', 'request_id': 'req-2cce1740-2aad-4df6-8002-1b47a7cd47db'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.293487 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b145c2c6c5447679b3cd5c3a3cd7d9e', 'request_id': 'req-2cce1740-2aad-4df6-8002-1b47a7cd47db'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.320369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] The token cache is being invalidate because project 9b145c2c6c5447679b3cd5c3a3cd7d9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:09.321066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.322408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.328273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1206/2374] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:09 2022] DELETE /identity/v3/projects/9b145c2c6c5447679b3cd5c3a3cd7d9e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:09.334545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8be02d-e653-4522-93d7-098f71917d35 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:09.356933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8be02d-e653-4522-93d7-098f71917d35 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:09.561322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:09.562549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:09.563065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:09.563573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] PATH_INFO: `/v3/users/ec400ab6c140471896af62d2f37f42bc` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:09.566592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec400ab6c140471896af62d2f37f42bc)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:09.570515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:09.654435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec400ab6c140471896af62d2f37f42bc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.663378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec400ab6c140471896af62d2f37f42bc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.664603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec400ab6c140471896af62d2f37f42bc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.684610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec400ab6c140471896af62d2f37f42bc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.685839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8636fd87-7a62-401e-b9ce-ed03b2b3b763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec400ab6c140471896af62d2f37f42bc'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:09.708376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1169/2375] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:09 2022] DELETE /identity/v3/users/ec400ab6c140471896af62d2f37f42bc => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:10.259324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:10.269510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:10.291078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce1740-2aad-4df6-8002-1b47a7cd47db admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:10.384616 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:10.386785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:10.387111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:10.387385 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] PATH_INFO: `/v3/projects/67df373e88c949f2865c164a49eb4cca` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:10.389053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67df373e88c949f2865c164a49eb4cca)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:10.391150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:10.414395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:10.459595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67df373e88c949f2865c164a49eb4cca', 'request_id': 'req-cf8413c5-6e6d-4359-91dd-6b3d132785f2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:10.508110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67df373e88c949f2865c164a49eb4cca', 'request_id': 'req-cf8413c5-6e6d-4359-91dd-6b3d132785f2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:10.523553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67df373e88c949f2865c164a49eb4cca', 'request_id': 'req-cf8413c5-6e6d-4359-91dd-6b3d132785f2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:10.539982 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] The token cache is being invalidate because project 67df373e88c949f2865c164a49eb4cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:10.540298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:10.540939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cf8413c5-6e6d-4359-91dd-6b3d132785f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:10.542617 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1207/2376] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:10 2022] DELETE /identity/v3/projects/67df373e88c949f2865c164a49eb4cca => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:10.548432 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60aab2-bfe9-4961-94e8-99560230c052 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:10.549241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60aab2-bfe9-4961-94e8-99560230c052 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:10.549678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60aab2-bfe9-4961-94e8-99560230c052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:10.576313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bd60aab2-bfe9-4961-94e8-99560230c052 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:10.577554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bd60aab2-bfe9-4961-94e8-99560230c052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:10.602870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:10.680629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1170/2377] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:10.690242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a54e8-09fc-46f2-8de0-50244aa22382 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:10.690928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a54e8-09fc-46f2-8de0-50244aa22382 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:10.690928 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a54e8-09fc-46f2-8de0-50244aa22382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:10.726927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ec8a54e8-09fc-46f2-8de0-50244aa22382 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:10.728739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ec8a54e8-09fc-46f2-8de0-50244aa22382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:10.741918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1208/2378] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:29:10.747914 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60aab2-bfe9-4961-94e8-99560230c052 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:10.763529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60aab2-bfe9-4961-94e8-99560230c052 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:10.770601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef9b06-6caf-42bb-8457-0315af3bbcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:10.771674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef9b06-6caf-42bb-8457-0315af3bbcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:10.772315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef9b06-6caf-42bb-8457-0315af3bbcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:10.772797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef9b06-6caf-42bb-8457-0315af3bbcc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:10.776565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef9b06-6caf-42bb-8457-0315af3bbcc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:29:10.777331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef9b06-6caf-42bb-8457-0315af3bbcc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:10.780656 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef9b06-6caf-42bb-8457-0315af3bbcc3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:10.805753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1171/2379] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:29:10.812572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a54e8-09fc-46f2-8de0-50244aa22382 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:10.825016 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a54e8-09fc-46f2-8de0-50244aa22382 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:10.833545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de1dd1-9c75-41d4-9ae5-4677a53b21e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:10.834515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de1dd1-9c75-41d4-9ae5-4677a53b21e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:10.834790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de1dd1-9c75-41d4-9ae5-4677a53b21e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:10.835061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de1dd1-9c75-41d4-9ae5-4677a53b21e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:10.835774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de1dd1-9c75-41d4-9ae5-4677a53b21e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:10.837819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de1dd1-9c75-41d4-9ae5-4677a53b21e8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:10.884233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1209/2380] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:10 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:10.889274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef9b06-6caf-42bb-8457-0315af3bbcc3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:10.900639 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef9b06-6caf-42bb-8457-0315af3bbcc3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:11.017774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07788d-24b6-47ba-ac48-74c25368c4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:11.024826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07788d-24b6-47ba-ac48-74c25368c4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:11.025435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07788d-24b6-47ba-ac48-74c25368c4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:11.028392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07788d-24b6-47ba-ac48-74c25368c4cc admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:11.029501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07788d-24b6-47ba-ac48-74c25368c4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:11.033560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07788d-24b6-47ba-ac48-74c25368c4cc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:11.077769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1172/2381] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:10 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:11.083978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de1dd1-9c75-41d4-9ae5-4677a53b21e8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:11.103454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de1dd1-9c75-41d4-9ae5-4677a53b21e8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:11.109544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6a46d-87b2-4e93-8096-81e5368442f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:11.114377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6a46d-87b2-4e93-8096-81e5368442f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:11.114691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6a46d-87b2-4e93-8096-81e5368442f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:11.114951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6a46d-87b2-4e93-8096-81e5368442f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:11.115896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6a46d-87b2-4e93-8096-81e5368442f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:11.126567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6a46d-87b2-4e93-8096-81e5368442f4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:11.157527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1210/2382] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:11.163641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07788d-24b6-47ba-ac48-74c25368c4cc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:11.203188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07788d-24b6-47ba-ac48-74c25368c4cc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:11.242446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183053e6-bf97-4c31-937d-7555c03a7f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:11.243678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183053e6-bf97-4c31-937d-7555c03a7f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:11.244179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183053e6-bf97-4c31-937d-7555c03a7f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:11.245169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183053e6-bf97-4c31-937d-7555c03a7f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:11.246609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183053e6-bf97-4c31-937d-7555c03a7f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:11.269605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183053e6-bf97-4c31-937d-7555c03a7f39 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:11.319680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1173/2383] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:11.321425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6a46d-87b2-4e93-8096-81e5368442f4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:11.336855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6a46d-87b2-4e93-8096-81e5368442f4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:11.353808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dceac-b308-4101-9141-55f613afcf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:11.355620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dceac-b308-4101-9141-55f613afcf92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:11.355909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dceac-b308-4101-9141-55f613afcf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:11.356220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dceac-b308-4101-9141-55f613afcf92 admin admin] PATH_INFO: `/v3/projects/12fda704f1a942d2b5ff191460b0b3ce/users/52032fb8f4a64627aa4a079454b693a8/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:11.398626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dceac-b308-4101-9141-55f613afcf92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12fda704f1a942d2b5ff191460b0b3ce, user_id=52032fb8f4a64627aa4a079454b693a8, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:11.406506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dceac-b308-4101-9141-55f613afcf92 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:11.457318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1211/2384] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:11 2022] PUT /identity/v3/projects/12fda704f1a942d2b5ff191460b0b3ce/users/52032fb8f4a64627aa4a079454b693a8/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:29:11.485663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c283cf-4934-4393-9e2f-5a7e34122630 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:11.486529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c283cf-4934-4393-9e2f-5a7e34122630 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:11.487048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c283cf-4934-4393-9e2f-5a7e34122630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:11.528663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d0c283cf-4934-4393-9e2f-5a7e34122630 None None] MFA Rules not processed for user `52032fb8f4a64627aa4a079454b693a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:11.529762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d0c283cf-4934-4393-9e2f-5a7e34122630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:11.645108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1174/2385] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:11.651943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dceac-b308-4101-9141-55f613afcf92 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:11.763030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dceac-b308-4101-9141-55f613afcf92 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:11.768545 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1d4fa-1eca-4aad-9999-c1754aa51bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:11.769377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1d4fa-1eca-4aad-9999-c1754aa51bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:11.769756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1d4fa-1eca-4aad-9999-c1754aa51bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:11.770110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1d4fa-1eca-4aad-9999-c1754aa51bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:11.770937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1d4fa-1eca-4aad-9999-c1754aa51bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:11.773007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1d4fa-1eca-4aad-9999-c1754aa51bb0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:11.804977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1212/2386] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:11 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:29:11.810429 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c283cf-4934-4393-9e2f-5a7e34122630 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:11.828493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c283cf-4934-4393-9e2f-5a7e34122630 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:11.844772 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:11.911171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4db43-cda9-4d65-9708-f38547034d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:11.911891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4db43-cda9-4d65-9708-f38547034d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:11.912175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4db43-cda9-4d65-9708-f38547034d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:11.912450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4db43-cda9-4d65-9708-f38547034d05 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:11.913271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4db43-cda9-4d65-9708-f38547034d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:11.915313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4db43-cda9-4d65-9708-f38547034d05 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:11.943114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1175/2387] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:11 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:11.947382 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1d4fa-1eca-4aad-9999-c1754aa51bb0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:11.960284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1d4fa-1eca-4aad-9999-c1754aa51bb0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:11.966906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39730fe1-54d8-4176-831c-b687a46ef307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:11.967714 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730fe1-54d8-4176-831c-b687a46ef307 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:11.968103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730fe1-54d8-4176-831c-b687a46ef307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:11.968476 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730fe1-54d8-4176-831c-b687a46ef307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:11.969368 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39730fe1-54d8-4176-831c-b687a46ef307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:11.971538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39730fe1-54d8-4176-831c-b687a46ef307 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:11.989922 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1213/2388] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:11.994447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4db43-cda9-4d65-9708-f38547034d05 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:12.006492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4db43-cda9-4d65-9708-f38547034d05 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:12.013311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040f226-f78a-4a08-92d7-d40beb132c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:12.014438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040f226-f78a-4a08-92d7-d40beb132c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:12.014873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040f226-f78a-4a08-92d7-d40beb132c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:12.015272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040f226-f78a-4a08-92d7-d40beb132c73 admin admin] PATH_INFO: `/v3/projects/14988b838dcf40eda97c456f1093eca5/users/849004148ed845cf8c7bd0814359c014/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:12.034318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040f226-f78a-4a08-92d7-d40beb132c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14988b838dcf40eda97c456f1093eca5, user_id=849004148ed845cf8c7bd0814359c014, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:12.037140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040f226-f78a-4a08-92d7-d40beb132c73 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:12.066303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1176/2389] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:11 2022] PUT /identity/v3/projects/14988b838dcf40eda97c456f1093eca5/users/849004148ed845cf8c7bd0814359c014/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:12.068314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39730fe1-54d8-4176-831c-b687a46ef307 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:12.081222 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39730fe1-54d8-4176-831c-b687a46ef307 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:12.092149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:12.185537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf342336-03ea-4bd8-99eb-1755ddc64788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:12.186446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342336-03ea-4bd8-99eb-1755ddc64788 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:12.186824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342336-03ea-4bd8-99eb-1755ddc64788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:12.187223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342336-03ea-4bd8-99eb-1755ddc64788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:12.188231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf342336-03ea-4bd8-99eb-1755ddc64788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:12.190213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf342336-03ea-4bd8-99eb-1755ddc64788 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:12.202114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1214/2390] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:12.206546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040f226-f78a-4a08-92d7-d40beb132c73 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:12.227557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040f226-f78a-4a08-92d7-d40beb132c73 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:12.236255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f1016-91bc-43e2-b1da-72c46bb5d883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:12.237850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f1016-91bc-43e2-b1da-72c46bb5d883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:12.238358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f1016-91bc-43e2-b1da-72c46bb5d883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:12.238844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f1016-91bc-43e2-b1da-72c46bb5d883 admin admin] PATH_INFO: `/v3/projects/14988b838dcf40eda97c456f1093eca5/users/849004148ed845cf8c7bd0814359c014/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:12.242718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f1016-91bc-43e2-b1da-72c46bb5d883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14988b838dcf40eda97c456f1093eca5, user_id=849004148ed845cf8c7bd0814359c014, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:12.247994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f1016-91bc-43e2-b1da-72c46bb5d883 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:12.265877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1177/2391] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:12 2022] PUT /identity/v3/projects/14988b838dcf40eda97c456f1093eca5/users/849004148ed845cf8c7bd0814359c014/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:12.281943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c9d3e-ee31-4b92-bdb7-a16e4846d455 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:12.282879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c9d3e-ee31-4b92-bdb7-a16e4846d455 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:12.283296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c9d3e-ee31-4b92-bdb7-a16e4846d455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:12.329442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ef7c9d3e-ee31-4b92-bdb7-a16e4846d455 None None] MFA Rules not processed for user `849004148ed845cf8c7bd0814359c014`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:12.331888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ef7c9d3e-ee31-4b92-bdb7-a16e4846d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:12.457816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1215/2392] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:12.475569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f1016-91bc-43e2-b1da-72c46bb5d883 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:12.478388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f38f1016-91bc-43e2-b1da-72c46bb5d883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:12.591126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f1016-91bc-43e2-b1da-72c46bb5d883 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:12.595787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de535b61-04b1-418f-a5a7-0ed60ebe9285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:12.596572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de535b61-04b1-418f-a5a7-0ed60ebe9285 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:12.596910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de535b61-04b1-418f-a5a7-0ed60ebe9285 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:12.597221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de535b61-04b1-418f-a5a7-0ed60ebe9285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:12.611014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de535b61-04b1-418f-a5a7-0ed60ebe9285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:12.613233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de535b61-04b1-418f-a5a7-0ed60ebe9285 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:12.625443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1178/2393] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:12.754003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c9d3e-ee31-4b92-bdb7-a16e4846d455 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:12.778468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c9d3e-ee31-4b92-bdb7-a16e4846d455 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:12.785880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42957a54-b364-4915-903f-aa8f18196d46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:12.787861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42957a54-b364-4915-903f-aa8f18196d46 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:12.789019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42957a54-b364-4915-903f-aa8f18196d46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:12.789774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42957a54-b364-4915-903f-aa8f18196d46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:12.997936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42957a54-b364-4915-903f-aa8f18196d46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:13.000139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42957a54-b364-4915-903f-aa8f18196d46 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:13.031721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1216/2394] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:18.748749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de535b61-04b1-418f-a5a7-0ed60ebe9285 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:18.895539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de535b61-04b1-418f-a5a7-0ed60ebe9285 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:18.909863 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:18.913336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:18.913890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:18.914399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] PATH_INFO: `/v3/users/52032fb8f4a64627aa4a079454b693a8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:18.918909 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52032fb8f4a64627aa4a079454b693a8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:18.924300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:18.951943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:19.007968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52032fb8f4a64627aa4a079454b693a8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:19.017888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52032fb8f4a64627aa4a079454b693a8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:19.018832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52032fb8f4a64627aa4a079454b693a8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:19.039642 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52032fb8f4a64627aa4a079454b693a8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:19.040850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52032fb8f4a64627aa4a079454b693a8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:19.064691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1179/2395] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:18 2022] DELETE /identity/v3/users/52032fb8f4a64627aa4a079454b693a8 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:19.069849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1394-f1d2-48f2-92b5-e3f410767252 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:19.070659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1394-f1d2-48f2-92b5-e3f410767252 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:19.070659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1394-f1d2-48f2-92b5-e3f410767252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:19.091950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a2cb1394-f1d2-48f2-92b5-e3f410767252 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:19.093301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a2cb1394-f1d2-48f2-92b5-e3f410767252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:19.136607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:19.210605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1217/2396] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:19.225148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:19.226992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:19.316617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba170d44-8cff-4470-9842-b3ed84ca8590 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:19.322114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb93876-79ad-4d0d-92a6-edc3577a35c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:19.322941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb93876-79ad-4d0d-92a6-edc3577a35c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:19.323333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb93876-79ad-4d0d-92a6-edc3577a35c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:19.323711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb93876-79ad-4d0d-92a6-edc3577a35c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:19.335644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb93876-79ad-4d0d-92a6-edc3577a35c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:19.337506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb93876-79ad-4d0d-92a6-edc3577a35c5 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:19.351661 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1180/2397] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:20.184310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1394-f1d2-48f2-92b5-e3f410767252 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:20.186539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a2cb1394-f1d2-48f2-92b5-e3f410767252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:20.207182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1394-f1d2-48f2-92b5-e3f410767252 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:20.216098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:20.217619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:20.218152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:20.218649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] PATH_INFO: `/v3/projects/12fda704f1a942d2b5ff191460b0b3ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:20.220971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12fda704f1a942d2b5ff191460b0b3ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:20.224927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:20.326453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12fda704f1a942d2b5ff191460b0b3ce', 'request_id': 'req-d7bd3ce8-cc79-4b81-a382-d3802b47e150'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.339595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12fda704f1a942d2b5ff191460b0b3ce', 'request_id': 'req-d7bd3ce8-cc79-4b81-a382-d3802b47e150'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.353633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12fda704f1a942d2b5ff191460b0b3ce', 'request_id': 'req-d7bd3ce8-cc79-4b81-a382-d3802b47e150'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.375531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] The token cache is being invalidate because project 12fda704f1a942d2b5ff191460b0b3ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:20.376040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.377156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d7bd3ce8-cc79-4b81-a382-d3802b47e150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.380248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1218/2398] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:20 2022] DELETE /identity/v3/projects/12fda704f1a942d2b5ff191460b0b3ce => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:29:20.386221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb93876-79ad-4d0d-92a6-edc3577a35c5 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:20.403834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb93876-79ad-4d0d-92a6-edc3577a35c5 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:20.486058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:20.486961 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:20.487246 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:20.487520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] PATH_INFO: `/v3/users/849004148ed845cf8c7bd0814359c014` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:20.488763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=849004148ed845cf8c7bd0814359c014)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:20.491163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:20.539730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:20.559885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '849004148ed845cf8c7bd0814359c014'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.571217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '849004148ed845cf8c7bd0814359c014'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.572440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '849004148ed845cf8c7bd0814359c014'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.598056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '849004148ed845cf8c7bd0814359c014'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.599266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '849004148ed845cf8c7bd0814359c014'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:20.630711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1181/2399] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:20 2022] DELETE /identity/v3/users/849004148ed845cf8c7bd0814359c014 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:20.669034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629dcc5-a0b9-4080-89fb-baf520e3899e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:20.669646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629dcc5-a0b9-4080-89fb-baf520e3899e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:20.670897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629dcc5-a0b9-4080-89fb-baf520e3899e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:20.697602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9629dcc5-a0b9-4080-89fb-baf520e3899e None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:20.699440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9629dcc5-a0b9-4080-89fb-baf520e3899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:20.799638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1219/2400] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 6 14:29:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:20.816020 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:20.818382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:20.956145 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9435a54-a324-4b70-8186-2afd188cae92 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:20.963130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b9433-19f1-4c0b-acd0-2456bfccbd7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:20.964111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b9433-19f1-4c0b-acd0-2456bfccbd7f service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:20.964397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b9433-19f1-4c0b-acd0-2456bfccbd7f service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:20.964673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b9433-19f1-4c0b-acd0-2456bfccbd7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:20.988695 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b9433-19f1-4c0b-acd0-2456bfccbd7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:20.990522 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b9433-19f1-4c0b-acd0-2456bfccbd7f service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:21.006883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1182/2401] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:29:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:29:21.281388 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629dcc5-a0b9-4080-89fb-baf520e3899e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:21.283660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9629dcc5-a0b9-4080-89fb-baf520e3899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:21.468166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:21.479174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629dcc5-a0b9-4080-89fb-baf520e3899e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:21.502081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:21.505943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:21.508209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:21.510008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] PATH_INFO: `/v3/projects/14988b838dcf40eda97c456f1093eca5` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:21.516048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14988b838dcf40eda97c456f1093eca5)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:21.528662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:21.728574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14988b838dcf40eda97c456f1093eca5', 'request_id': 'req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:21.757254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14988b838dcf40eda97c456f1093eca5', 'request_id': 'req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:21.774927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14988b838dcf40eda97c456f1093eca5', 'request_id': 'req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:21.797556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] The token cache is being invalidate because project 14988b838dcf40eda97c456f1093eca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:21.797872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:21.799211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e977a5df-cce8-4a7d-a27b-5d225e7ca7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:21.800673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1220/2402] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:21 2022] DELETE /identity/v3/projects/14988b838dcf40eda97c456f1093eca5 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:21.807567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc9787-b855-4fd6-b704-e7713c5696a1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:21.808949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc9787-b855-4fd6-b704-e7713c5696a1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:21.809414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc9787-b855-4fd6-b704-e7713c5696a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:21.840474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ddfc9787-b855-4fd6-b704-e7713c5696a1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:21.842853 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ddfc9787-b855-4fd6-b704-e7713c5696a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:21.944203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1183/2403] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:21.951048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5ef58-42e9-47cb-81bd-396904091948 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:21.951611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5ef58-42e9-47cb-81bd-396904091948 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:21.951910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5ef58-42e9-47cb-81bd-396904091948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:21.984477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-38e5ef58-42e9-47cb-81bd-396904091948 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:21.985870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-38e5ef58-42e9-47cb-81bd-396904091948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:21.995482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1221/2404] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:22.001425 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc9787-b855-4fd6-b704-e7713c5696a1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:22.013949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc9787-b855-4fd6-b704-e7713c5696a1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:22.020106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39af2d7-fd1b-4f79-ab4a-ce0e084ea044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:22.020902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39af2d7-fd1b-4f79-ab4a-ce0e084ea044 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:22.021555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39af2d7-fd1b-4f79-ab4a-ce0e084ea044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:22.021941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39af2d7-fd1b-4f79-ab4a-ce0e084ea044 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:22.022815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39af2d7-fd1b-4f79-ab4a-ce0e084ea044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:29:22.023963 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39af2d7-fd1b-4f79-ab4a-ce0e084ea044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:22.027323 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39af2d7-fd1b-4f79-ab4a-ce0e084ea044 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:22.058004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1184/2405] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:29:22.059495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5ef58-42e9-47cb-81bd-396904091948 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:22.073095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5ef58-42e9-47cb-81bd-396904091948 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:22.090134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996506-265a-4ae4-96ac-34e83864b206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:22.090878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996506-265a-4ae4-96ac-34e83864b206 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:22.091157 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996506-265a-4ae4-96ac-34e83864b206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:22.091437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996506-265a-4ae4-96ac-34e83864b206 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:22.092307 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74996506-265a-4ae4-96ac-34e83864b206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:22.094615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74996506-265a-4ae4-96ac-34e83864b206 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:22.163958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1222/2406] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:22 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:22.168873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39af2d7-fd1b-4f79-ab4a-ce0e084ea044 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:22.198017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39af2d7-fd1b-4f79-ab4a-ce0e084ea044 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:22.282376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:22.410039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0400c8-9e41-4231-a2b3-dca9be64f5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:22.411130 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0400c8-9e41-4231-a2b3-dca9be64f5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:22.411507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0400c8-9e41-4231-a2b3-dca9be64f5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:22.411896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0400c8-9e41-4231-a2b3-dca9be64f5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:22.413681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0400c8-9e41-4231-a2b3-dca9be64f5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:22.422008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0400c8-9e41-4231-a2b3-dca9be64f5d9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:22.476976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1185/2407] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:22 2022] POST /identity/v3/users => generated 288 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:29:22.482166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996506-265a-4ae4-96ac-34e83864b206 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:22.496852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996506-265a-4ae4-96ac-34e83864b206 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:22.517241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c8be2-ec93-4e91-b023-fc0e4c6557cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:22.519568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c8be2-ec93-4e91-b023-fc0e4c6557cb admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:22.520475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c8be2-ec93-4e91-b023-fc0e4c6557cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:22.521149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c8be2-ec93-4e91-b023-fc0e4c6557cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:22.522637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c8be2-ec93-4e91-b023-fc0e4c6557cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:22.529248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c8be2-ec93-4e91-b023-fc0e4c6557cb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:22.552293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1223/2408] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:22.555763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0400c8-9e41-4231-a2b3-dca9be64f5d9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:22.595115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0400c8-9e41-4231-a2b3-dca9be64f5d9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:22.604911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4a1dd-3948-4076-a09c-9e2089505fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:22.606225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4a1dd-3948-4076-a09c-9e2089505fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:22.606843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4a1dd-3948-4076-a09c-9e2089505fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:22.607441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4a1dd-3948-4076-a09c-9e2089505fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:22.608608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4a1dd-3948-4076-a09c-9e2089505fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:22.611290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4a1dd-3948-4076-a09c-9e2089505fb2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:22.629420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1186/2409] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:22.634783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c8be2-ec93-4e91-b023-fc0e4c6557cb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:22.656719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c8be2-ec93-4e91-b023-fc0e4c6557cb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:22.668023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2365e6-5a16-4d9c-b57d-12aab267f879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:22.669847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2365e6-5a16-4d9c-b57d-12aab267f879 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:22.670354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2365e6-5a16-4d9c-b57d-12aab267f879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:22.670873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2365e6-5a16-4d9c-b57d-12aab267f879 admin admin] PATH_INFO: `/v3/projects/be09e1dce9104a539b93e6e3adadedfb/users/34c60a47e91f471f81708c755aa9ec3d/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:22.703399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2365e6-5a16-4d9c-b57d-12aab267f879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be09e1dce9104a539b93e6e3adadedfb, user_id=34c60a47e91f471f81708c755aa9ec3d, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:22.710538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2365e6-5a16-4d9c-b57d-12aab267f879 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:22.773448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1224/2410] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:22 2022] PUT /identity/v3/projects/be09e1dce9104a539b93e6e3adadedfb/users/34c60a47e91f471f81708c755aa9ec3d/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:29:22.793417 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fe525-0330-4355-9261-4db424d494a0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:22.793797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fe525-0330-4355-9261-4db424d494a0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:22.794204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fe525-0330-4355-9261-4db424d494a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:22.829176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-de0fe525-0330-4355-9261-4db424d494a0 None None] MFA Rules not processed for user `34c60a47e91f471f81708c755aa9ec3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:22.831220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-de0fe525-0330-4355-9261-4db424d494a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:22.918059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1187/2411] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:29:22.929929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2365e6-5a16-4d9c-b57d-12aab267f879 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:22.932856 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4c2365e6-5a16-4d9c-b57d-12aab267f879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:23.040604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2365e6-5a16-4d9c-b57d-12aab267f879 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:23.047135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be234ef-19ff-4bff-8a80-87536f0ee186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:23.048094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be234ef-19ff-4bff-8a80-87536f0ee186 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:23.048552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be234ef-19ff-4bff-8a80-87536f0ee186 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:23.049034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be234ef-19ff-4bff-8a80-87536f0ee186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:23.067003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be234ef-19ff-4bff-8a80-87536f0ee186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:23.069847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be234ef-19ff-4bff-8a80-87536f0ee186 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:23.094805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1225/2412] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:29:27.442781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fe525-0330-4355-9261-4db424d494a0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:27.560837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fe525-0330-4355-9261-4db424d494a0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:27.583301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:27.584917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:27.585466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:27.586112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] PATH_INFO: `/v3/users/34c60a47e91f471f81708c755aa9ec3d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:27.590415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c60a47e91f471f81708c755aa9ec3d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:27.595170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:27.668678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:27.681140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c60a47e91f471f81708c755aa9ec3d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:27.701433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c60a47e91f471f81708c755aa9ec3d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:27.702631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c60a47e91f471f81708c755aa9ec3d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:27.727727 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c60a47e91f471f81708c755aa9ec3d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:27.730068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c60a47e91f471f81708c755aa9ec3d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:27.769688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1188/2413] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:27 2022] DELETE /identity/v3/users/34c60a47e91f471f81708c755aa9ec3d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:27.772460 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a10b5-c6d4-4427-9083-fabcb566d2e7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:27.773059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a10b5-c6d4-4427-9083-fabcb566d2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:27.773319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a10b5-c6d4-4427-9083-fabcb566d2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:27.821329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bc8a10b5-c6d4-4427-9083-fabcb566d2e7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:27.823310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bc8a10b5-c6d4-4427-9083-fabcb566d2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:27.852267 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:27.957512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:27.959358 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:27.961893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1226/2414] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:28.092438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7e4f2-6356-4e39-8622-f33938560ebd admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:28.097513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ae975-7588-41c0-b2d7-1182fc89608c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:28.098243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ae975-7588-41c0-b2d7-1182fc89608c service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:28.098592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ae975-7588-41c0-b2d7-1182fc89608c service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:28.098860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ae975-7588-41c0-b2d7-1182fc89608c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:28.113588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ae975-7588-41c0-b2d7-1182fc89608c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:28.116121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ae975-7588-41c0-b2d7-1182fc89608c service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:28.137552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1189/2415] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:29:28.802652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a10b5-c6d4-4427-9083-fabcb566d2e7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:28.805045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bc8a10b5-c6d4-4427-9083-fabcb566d2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:28.878627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a10b5-c6d4-4427-9083-fabcb566d2e7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:28.902554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:28.904122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:28.904564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:28.905041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] PATH_INFO: `/v3/projects/be09e1dce9104a539b93e6e3adadedfb` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:28.912304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] RBAC: Authorizing `identity:delete_project(project_id=be09e1dce9104a539b93e6e3adadedfb)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:28.917239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:29.040495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be09e1dce9104a539b93e6e3adadedfb', 'request_id': 'req-350279bf-f6e1-4221-aef2-895c374d743d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:29.063316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be09e1dce9104a539b93e6e3adadedfb', 'request_id': 'req-350279bf-f6e1-4221-aef2-895c374d743d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:29.080050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be09e1dce9104a539b93e6e3adadedfb', 'request_id': 'req-350279bf-f6e1-4221-aef2-895c374d743d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:29.099445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] The token cache is being invalidate because project be09e1dce9104a539b93e6e3adadedfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:29.100049 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:29.101270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-350279bf-f6e1-4221-aef2-895c374d743d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:29.118490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1227/2416] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:28 2022] DELETE /identity/v3/projects/be09e1dce9104a539b93e6e3adadedfb => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:29.120759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85135e8d-26e0-463a-89c0-ba040b35bca8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.122942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85135e8d-26e0-463a-89c0-ba040b35bca8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.126364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85135e8d-26e0-463a-89c0-ba040b35bca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.157272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-85135e8d-26e0-463a-89c0-ba040b35bca8 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:29.158577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-85135e8d-26e0-463a-89c0-ba040b35bca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:29.264826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1190/2417] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:29.277463 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab63f5-3c0a-46b6-823c-e66a22eee45c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.277777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab63f5-3c0a-46b6-823c-e66a22eee45c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.278050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab63f5-3c0a-46b6-823c-e66a22eee45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.299981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cbab63f5-3c0a-46b6-823c-e66a22eee45c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:29.301285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cbab63f5-3c0a-46b6-823c-e66a22eee45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:29.310590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1228/2418] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:29.315483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85135e8d-26e0-463a-89c0-ba040b35bca8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:29.325997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85135e8d-26e0-463a-89c0-ba040b35bca8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:29.331529 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092e87-c464-4928-ad63-b0378dae04ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:29.332234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092e87-c464-4928-ad63-b0378dae04ae admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.332517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092e87-c464-4928-ad63-b0378dae04ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.332790 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf092e87-c464-4928-ad63-b0378dae04ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.333537 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf092e87-c464-4928-ad63-b0378dae04ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:29:29.333880 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf092e87-c464-4928-ad63-b0378dae04ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:29.335586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf092e87-c464-4928-ad63-b0378dae04ae admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:29.356812 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1191/2419] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:29:29.360171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab63f5-3c0a-46b6-823c-e66a22eee45c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:29.371541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab63f5-3c0a-46b6-823c-e66a22eee45c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:29.377583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d52bfe-3811-4c43-aa69-f5f684aa7769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:29.378200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d52bfe-3811-4c43-aa69-f5f684aa7769 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.378480 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d52bfe-3811-4c43-aa69-f5f684aa7769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.378754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d52bfe-3811-4c43-aa69-f5f684aa7769 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.379475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d52bfe-3811-4c43-aa69-f5f684aa7769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:29.381433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d52bfe-3811-4c43-aa69-f5f684aa7769 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:29.414126 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1229/2420] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:29 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:29:29.420046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092e87-c464-4928-ad63-b0378dae04ae admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:29.431306 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf092e87-c464-4928-ad63-b0378dae04ae admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:29.492493 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:29.514042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237568e0-934e-4bc0-bf4f-a3f38dd9ed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:29.515076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237568e0-934e-4bc0-bf4f-a3f38dd9ed10 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.515456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237568e0-934e-4bc0-bf4f-a3f38dd9ed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.515829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237568e0-934e-4bc0-bf4f-a3f38dd9ed10 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.516724 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237568e0-934e-4bc0-bf4f-a3f38dd9ed10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:29.519164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237568e0-934e-4bc0-bf4f-a3f38dd9ed10 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:29.545455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1192/2421] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:29 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:29:29.552968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d52bfe-3811-4c43-aa69-f5f684aa7769 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:29.565448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d52bfe-3811-4c43-aa69-f5f684aa7769 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:29.570418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e74f97-6a63-4a2b-9cf6-ec98cb02f41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:29.571107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e74f97-6a63-4a2b-9cf6-ec98cb02f41f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.571392 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e74f97-6a63-4a2b-9cf6-ec98cb02f41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.571669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e74f97-6a63-4a2b-9cf6-ec98cb02f41f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.572429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e74f97-6a63-4a2b-9cf6-ec98cb02f41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:29.574394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e74f97-6a63-4a2b-9cf6-ec98cb02f41f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:29.586310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1230/2422] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:29:29.592868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237568e0-934e-4bc0-bf4f-a3f38dd9ed10 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:29.603177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237568e0-934e-4bc0-bf4f-a3f38dd9ed10 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:29.609308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25810b71-7358-4bd2-a3ad-f840b9ebd85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:29.610036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810b71-7358-4bd2-a3ad-f840b9ebd85f admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.610323 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810b71-7358-4bd2-a3ad-f840b9ebd85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.610596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810b71-7358-4bd2-a3ad-f840b9ebd85f admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.611330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25810b71-7358-4bd2-a3ad-f840b9ebd85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:29.612956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25810b71-7358-4bd2-a3ad-f840b9ebd85f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:29.625176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1193/2423] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:29.630277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e74f97-6a63-4a2b-9cf6-ec98cb02f41f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:29.641925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e74f97-6a63-4a2b-9cf6-ec98cb02f41f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:29.647619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619a32f-3763-486a-a990-eb031b6afe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:29.648641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619a32f-3763-486a-a990-eb031b6afe70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.649025 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619a32f-3763-486a-a990-eb031b6afe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.649298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619a32f-3763-486a-a990-eb031b6afe70 admin admin] PATH_INFO: `/v3/projects/af458b19201b4328b0f05713d9f36936/users/4b812cc8de79416d945532963ea98424/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.667572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619a32f-3763-486a-a990-eb031b6afe70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af458b19201b4328b0f05713d9f36936, user_id=4b812cc8de79416d945532963ea98424, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:29.670339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619a32f-3763-486a-a990-eb031b6afe70 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:29.685682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1231/2424] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:29 2022] PUT /identity/v3/projects/af458b19201b4328b0f05713d9f36936/users/4b812cc8de79416d945532963ea98424/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:29.697810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fd6c1-3a74-4fa0-b0c9-c44e15fdbb1d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.698199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fd6c1-3a74-4fa0-b0c9-c44e15fdbb1d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.698671 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fd6c1-3a74-4fa0-b0c9-c44e15fdbb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.736063 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-323fd6c1-3a74-4fa0-b0c9-c44e15fdbb1d None None] MFA Rules not processed for user `4b812cc8de79416d945532963ea98424`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:29.737473 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-323fd6c1-3a74-4fa0-b0c9-c44e15fdbb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:29.829353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1194/2425] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:29.838488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619a32f-3763-486a-a990-eb031b6afe70 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:29.946218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:29.968631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619a32f-3763-486a-a990-eb031b6afe70 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:29.973639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51acc9b-c20d-42f5-ace6-d97bddafec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:29.974384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51acc9b-c20d-42f5-ace6-d97bddafec3a admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:29.974666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51acc9b-c20d-42f5-ace6-d97bddafec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:29.974940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51acc9b-c20d-42f5-ace6-d97bddafec3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:29.975649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51acc9b-c20d-42f5-ace6-d97bddafec3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:29.977618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51acc9b-c20d-42f5-ace6-d97bddafec3a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:30.011166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1232/2426] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:29 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:30.016899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fd6c1-3a74-4fa0-b0c9-c44e15fdbb1d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:30.029601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fd6c1-3a74-4fa0-b0c9-c44e15fdbb1d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:30.117337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fd41a-fd72-43b9-8769-a987ee514e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:30.118449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fd41a-fd72-43b9-8769-a987ee514e44 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:30.118911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fd41a-fd72-43b9-8769-a987ee514e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:30.119446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fd41a-fd72-43b9-8769-a987ee514e44 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:30.120735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fd41a-fd72-43b9-8769-a987ee514e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:30.124617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fd41a-fd72-43b9-8769-a987ee514e44 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:30.153238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1195/2427] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:30 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:29:30.157383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51acc9b-c20d-42f5-ace6-d97bddafec3a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:30.170142 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51acc9b-c20d-42f5-ace6-d97bddafec3a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:30.175296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d491fc-c070-423c-8055-5a04e766b16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:30.175981 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d491fc-c070-423c-8055-5a04e766b16a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:30.176320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d491fc-c070-423c-8055-5a04e766b16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:30.176543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d491fc-c070-423c-8055-5a04e766b16a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:30.177283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d491fc-c070-423c-8055-5a04e766b16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:30.178967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d491fc-c070-423c-8055-5a04e766b16a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:30.192139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1233/2428] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:30.196585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fd41a-fd72-43b9-8769-a987ee514e44 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:30.208142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fd41a-fd72-43b9-8769-a987ee514e44 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:30.215187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1652c-18a6-43bb-b221-d84c272872f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:30.215890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1652c-18a6-43bb-b221-d84c272872f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:30.216177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1652c-18a6-43bb-b221-d84c272872f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:30.216456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1652c-18a6-43bb-b221-d84c272872f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:30.217210 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1652c-18a6-43bb-b221-d84c272872f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:30.218921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1652c-18a6-43bb-b221-d84c272872f2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:30.236636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1196/2429] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:29:30.237821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d491fc-c070-423c-8055-5a04e766b16a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:30.249958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d491fc-c070-423c-8055-5a04e766b16a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:30.256244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cb975-061d-4685-b0e8-5df946e6db8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:30.257905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cb975-061d-4685-b0e8-5df946e6db8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:30.258403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cb975-061d-4685-b0e8-5df946e6db8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:30.258887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cb975-061d-4685-b0e8-5df946e6db8c admin admin] PATH_INFO: `/v3/projects/afeefd51056443789b0eb4de21dc6e40/users/728b609067564f13b2463df11937c490/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:30.293722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0cb975-061d-4685-b0e8-5df946e6db8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=afeefd51056443789b0eb4de21dc6e40, user_id=728b609067564f13b2463df11937c490, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:30.299061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0cb975-061d-4685-b0e8-5df946e6db8c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:30.332214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1234/2430] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:30 2022] PUT /identity/v3/projects/afeefd51056443789b0eb4de21dc6e40/users/728b609067564f13b2463df11937c490/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:29:30.344310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9affe9-2683-409c-b6fc-8ea7666bc39d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:30.344612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9affe9-2683-409c-b6fc-8ea7666bc39d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:30.344887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9affe9-2683-409c-b6fc-8ea7666bc39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:30.384021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bd9affe9-2683-409c-b6fc-8ea7666bc39d None None] MFA Rules not processed for user `728b609067564f13b2463df11937c490`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:30.386404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bd9affe9-2683-409c-b6fc-8ea7666bc39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:30.424166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:30.480574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1197/2431] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:30.485474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cb975-061d-4685-b0e8-5df946e6db8c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:30.567190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cb975-061d-4685-b0e8-5df946e6db8c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:30.585857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea435c-f620-4956-9454-f83f5cdaf7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:30.585857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea435c-f620-4956-9454-f83f5cdaf7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:30.585857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea435c-f620-4956-9454-f83f5cdaf7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:30.585857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea435c-f620-4956-9454-f83f5cdaf7c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:30.588334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea435c-f620-4956-9454-f83f5cdaf7c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:30.588334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea435c-f620-4956-9454-f83f5cdaf7c4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:30.623042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1235/2432] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:30 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:29:30.629322 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9affe9-2683-409c-b6fc-8ea7666bc39d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:30.647498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9affe9-2683-409c-b6fc-8ea7666bc39d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:30.759562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39def102-3df9-43bf-a388-889d364d313a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:30.761219 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39def102-3df9-43bf-a388-889d364d313a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:30.761749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39def102-3df9-43bf-a388-889d364d313a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:30.762261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39def102-3df9-43bf-a388-889d364d313a admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:30.763577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39def102-3df9-43bf-a388-889d364d313a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:30.767471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39def102-3df9-43bf-a388-889d364d313a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:30.803889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1198/2433] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:30 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:30.809386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea435c-f620-4956-9454-f83f5cdaf7c4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:30.825033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea435c-f620-4956-9454-f83f5cdaf7c4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:30.830319 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140896b-ab57-4f9b-88dd-5a88b32a7385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:30.831020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140896b-ab57-4f9b-88dd-5a88b32a7385 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:30.831335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140896b-ab57-4f9b-88dd-5a88b32a7385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:30.831612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140896b-ab57-4f9b-88dd-5a88b32a7385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:30.832584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140896b-ab57-4f9b-88dd-5a88b32a7385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:30.834184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140896b-ab57-4f9b-88dd-5a88b32a7385 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:30.848914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1236/2434] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:30.854563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39def102-3df9-43bf-a388-889d364d313a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:30.865942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39def102-3df9-43bf-a388-889d364d313a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:30.872244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cd284-c935-4dea-aa28-013b86b0a812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:30.873121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cd284-c935-4dea-aa28-013b86b0a812 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:30.873470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cd284-c935-4dea-aa28-013b86b0a812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:30.873769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cd284-c935-4dea-aa28-013b86b0a812 admin admin] PATH_INFO: `/v3/projects/cbfb862edafb40d7bd49a7773fd6f45a/users/8a0410fddae24b8ba1068cf7d3100ce5/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:30.893910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cd284-c935-4dea-aa28-013b86b0a812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfb862edafb40d7bd49a7773fd6f45a, user_id=8a0410fddae24b8ba1068cf7d3100ce5, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:30.896723 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cd284-c935-4dea-aa28-013b86b0a812 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:30.919596 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1199/2435] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:30 2022] PUT /identity/v3/projects/cbfb862edafb40d7bd49a7773fd6f45a/users/8a0410fddae24b8ba1068cf7d3100ce5/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:30.924024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140896b-ab57-4f9b-88dd-5a88b32a7385 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:30.935482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140896b-ab57-4f9b-88dd-5a88b32a7385 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:31.025093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5642059-62cd-4443-9836-a944227bcda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:31.025831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5642059-62cd-4443-9836-a944227bcda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:31.026162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5642059-62cd-4443-9836-a944227bcda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:31.026427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5642059-62cd-4443-9836-a944227bcda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:31.027153 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5642059-62cd-4443-9836-a944227bcda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:31.028816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5642059-62cd-4443-9836-a944227bcda3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:31.040458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1237/2436] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:29:31.046077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cd284-c935-4dea-aa28-013b86b0a812 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:31.063955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cd284-c935-4dea-aa28-013b86b0a812 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:31.069917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d11b-583f-4ca4-95d8-e774836e34c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:31.071538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d11b-583f-4ca4-95d8-e774836e34c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:31.071538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d11b-583f-4ca4-95d8-e774836e34c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:31.071538 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43d11b-583f-4ca4-95d8-e774836e34c4 admin admin] PATH_INFO: `/v3/projects/cbfb862edafb40d7bd49a7773fd6f45a/users/8a0410fddae24b8ba1068cf7d3100ce5/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:31.074087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d11b-583f-4ca4-95d8-e774836e34c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfb862edafb40d7bd49a7773fd6f45a, user_id=8a0410fddae24b8ba1068cf7d3100ce5, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:31.076953 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43d11b-583f-4ca4-95d8-e774836e34c4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:31.098293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1200/2437] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:31 2022] PUT /identity/v3/projects/cbfb862edafb40d7bd49a7773fd6f45a/users/8a0410fddae24b8ba1068cf7d3100ce5/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:31.192127 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938ec2e-9d1f-41d4-b5c0-d66a3ecdb596 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:31.192421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938ec2e-9d1f-41d4-b5c0-d66a3ecdb596 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:31.192695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938ec2e-9d1f-41d4-b5c0-d66a3ecdb596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:31.206884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:31.241356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e938ec2e-9d1f-41d4-b5c0-d66a3ecdb596 None None] MFA Rules not processed for user `8a0410fddae24b8ba1068cf7d3100ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:31.244699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e938ec2e-9d1f-41d4-b5c0-d66a3ecdb596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:31.356758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1238/2438] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:31.369685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d11b-583f-4ca4-95d8-e774836e34c4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:31.371536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9b43d11b-583f-4ca4-95d8-e774836e34c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:31.438670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43d11b-583f-4ca4-95d8-e774836e34c4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:31.442706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734a3fa-cb59-4631-a647-778dbb8ee644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:31.443385 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a3fa-cb59-4631-a647-778dbb8ee644 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:31.443670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a3fa-cb59-4631-a647-778dbb8ee644 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:31.443948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a3fa-cb59-4631-a647-778dbb8ee644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:31.482251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:31.516800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734a3fa-cb59-4631-a647-778dbb8ee644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:31.518362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734a3fa-cb59-4631-a647-778dbb8ee644 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:31.530721 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1201/2439] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:33.193610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938ec2e-9d1f-41d4-b5c0-d66a3ecdb596 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:33.195310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e938ec2e-9d1f-41d4-b5c0-d66a3ecdb596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:33.209762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938ec2e-9d1f-41d4-b5c0-d66a3ecdb596 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:33.215005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d671a-e311-44a7-b40a-6b13228f907c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:33.215679 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d671a-e311-44a7-b40a-6b13228f907c service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:33.215959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d671a-e311-44a7-b40a-6b13228f907c service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:33.216235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d671a-e311-44a7-b40a-6b13228f907c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:33.287517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d671a-e311-44a7-b40a-6b13228f907c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:33.289168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d671a-e311-44a7-b40a-6b13228f907c service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:33.300556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1239/2440] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:29:34.084799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734a3fa-cb59-4631-a647-778dbb8ee644 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:34.095017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734a3fa-cb59-4631-a647-778dbb8ee644 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:34.099521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a23f7f-cf9a-4f29-8a14-cba46e1120cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:34.100281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a23f7f-cf9a-4f29-8a14-cba46e1120cc service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:34.100601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a23f7f-cf9a-4f29-8a14-cba46e1120cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:34.100913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a23f7f-cf9a-4f29-8a14-cba46e1120cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:34.112177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a23f7f-cf9a-4f29-8a14-cba46e1120cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:34.113888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a23f7f-cf9a-4f29-8a14-cba46e1120cc service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:34.126161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1202/2441] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:38.372723 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d671a-e311-44a7-b40a-6b13228f907c service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:38.444113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d671a-e311-44a7-b40a-6b13228f907c service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:38.449531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:38.450600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:38.450993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:38.451378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] PATH_INFO: `/v3/users/4b812cc8de79416d945532963ea98424` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:38.452936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b812cc8de79416d945532963ea98424)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:38.455182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:38.518840 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:38.544787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b812cc8de79416d945532963ea98424'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:38.576309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b812cc8de79416d945532963ea98424'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:38.577800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b812cc8de79416d945532963ea98424'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:38.580309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ddcef-f3ae-4c25-b8ef-1fe10d1a2832 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:38.581305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ddcef-f3ae-4c25-b8ef-1fe10d1a2832 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:38.581591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ddcef-f3ae-4c25-b8ef-1fe10d1a2832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:38.597384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b812cc8de79416d945532963ea98424'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:38.605857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ef3ddcef-f3ae-4c25-b8ef-1fe10d1a2832 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:38.607483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ef3ddcef-f3ae-4c25-b8ef-1fe10d1a2832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:38.624885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2a32dd27-4ecd-4e93-bb25-ad8797ab4c50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b812cc8de79416d945532963ea98424'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:38.634283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1240/2442] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:38 2022] DELETE /identity/v3/users/4b812cc8de79416d945532963ea98424 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:38.639293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0116b7b-8199-4319-b22f-a2bf11aacdae None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:38.639602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0116b7b-8199-4319-b22f-a2bf11aacdae None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:38.639990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0116b7b-8199-4319-b22f-a2bf11aacdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:38.660334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c0116b7b-8199-4319-b22f-a2bf11aacdae None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:38.661582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c0116b7b-8199-4319-b22f-a2bf11aacdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:38.708266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1203/2443] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 6 14:29:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:29:38.730771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ddcef-f3ae-4c25-b8ef-1fe10d1a2832 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:38.733228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ef3ddcef-f3ae-4c25-b8ef-1fe10d1a2832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:38.751875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1241/2444] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:29:38.768158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0116b7b-8199-4319-b22f-a2bf11aacdae None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:38.770628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c0116b7b-8199-4319-b22f-a2bf11aacdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:38.833984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ddcef-f3ae-4c25-b8ef-1fe10d1a2832 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:38.852070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0116b7b-8199-4319-b22f-a2bf11aacdae None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:38.857330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c6eaa-de98-4b42-8f5c-8698159c1278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:38.858224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c6eaa-de98-4b42-8f5c-8698159c1278 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:38.858516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c6eaa-de98-4b42-8f5c-8698159c1278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:38.858810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c6eaa-de98-4b42-8f5c-8698159c1278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:38.869689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c6eaa-de98-4b42-8f5c-8698159c1278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:38.871260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c6eaa-de98-4b42-8f5c-8698159c1278 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:38.883917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1242/2445] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:38.906019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3532b-87ea-4e70-b2bd-a1b64dd391af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:38.906703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3532b-87ea-4e70-b2bd-a1b64dd391af service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:38.907142 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3532b-87ea-4e70-b2bd-a1b64dd391af service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:38.907646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3532b-87ea-4e70-b2bd-a1b64dd391af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:38.919666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:38.999810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3532b-87ea-4e70-b2bd-a1b64dd391af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:39.001531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3532b-87ea-4e70-b2bd-a1b64dd391af service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:39.012131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1204/2446] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:29:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 14:29:39.263677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c6eaa-de98-4b42-8f5c-8698159c1278 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:39.265183 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-360c6eaa-de98-4b42-8f5c-8698159c1278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:39.283979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c6eaa-de98-4b42-8f5c-8698159c1278 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:39.291589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:39.293181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:39.293467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:39.293780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] PATH_INFO: `/v3/projects/af458b19201b4328b0f05713d9f36936` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:39.296439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af458b19201b4328b0f05713d9f36936)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:39.300325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:39.394121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:39.403683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af458b19201b4328b0f05713d9f36936', 'request_id': 'req-3d6b4e52-f76f-46aa-a516-ea7ab7568195'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.419957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af458b19201b4328b0f05713d9f36936', 'request_id': 'req-3d6b4e52-f76f-46aa-a516-ea7ab7568195'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.431054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af458b19201b4328b0f05713d9f36936', 'request_id': 'req-3d6b4e52-f76f-46aa-a516-ea7ab7568195'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.450932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] The token cache is being invalidate because project af458b19201b4328b0f05713d9f36936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:39.451419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.452326 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.455473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1243/2447] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:39 2022] DELETE /identity/v3/projects/af458b19201b4328b0f05713d9f36936 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:29:39.460833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3532b-87ea-4e70-b2bd-a1b64dd391af service swift] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:39.476803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3532b-87ea-4e70-b2bd-a1b64dd391af service swift] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:39.555027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:39.555978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:39.556889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:39.557171 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] PATH_INFO: `/v3/users/728b609067564f13b2463df11937c490` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:39.558487 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=728b609067564f13b2463df11937c490)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:39.560455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:39.625337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '728b609067564f13b2463df11937c490'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.633774 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '728b609067564f13b2463df11937c490'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.634566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '728b609067564f13b2463df11937c490'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.650373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '728b609067564f13b2463df11937c490'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.651242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '728b609067564f13b2463df11937c490'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:39.666525 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1205/2448] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:39 2022] DELETE /identity/v3/users/728b609067564f13b2463df11937c490 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:40.120889 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:40.122379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:40.136433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4e52-f76f-46aa-a516-ea7ab7568195 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:40.226451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:40.227857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:40.229579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:40.230353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] PATH_INFO: `/v3/projects/afeefd51056443789b0eb4de21dc6e40` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:40.232458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afeefd51056443789b0eb4de21dc6e40)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:40.236843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:40.318945 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afeefd51056443789b0eb4de21dc6e40', 'request_id': 'req-96d25977-a0b7-4124-9a86-f5543782b9d4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.333915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afeefd51056443789b0eb4de21dc6e40', 'request_id': 'req-96d25977-a0b7-4124-9a86-f5543782b9d4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.347384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afeefd51056443789b0eb4de21dc6e40', 'request_id': 'req-96d25977-a0b7-4124-9a86-f5543782b9d4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.370609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] The token cache is being invalidate because project afeefd51056443789b0eb4de21dc6e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:40.371121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.372406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.375227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1244/2449] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:40 2022] DELETE /identity/v3/projects/afeefd51056443789b0eb4de21dc6e40 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:40.381764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:40.383587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:40.405351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00ffe7-6f6a-4db9-a26d-c8c780fc84cb admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:40.449494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:40.512087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:40.513062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:40.513370 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:40.513631 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] PATH_INFO: `/v3/users/8a0410fddae24b8ba1068cf7d3100ce5` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:40.515135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0410fddae24b8ba1068cf7d3100ce5)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:40.517240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:40.605827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0410fddae24b8ba1068cf7d3100ce5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.622570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0410fddae24b8ba1068cf7d3100ce5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.624709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0410fddae24b8ba1068cf7d3100ce5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.650547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0410fddae24b8ba1068cf7d3100ce5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.651712 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a89f8692-843b-47e5-af77-980fc4fa2d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0410fddae24b8ba1068cf7d3100ce5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:40.675503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1206/2450] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:40 2022] DELETE /identity/v3/users/8a0410fddae24b8ba1068cf7d3100ce5 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:41.077983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:41.080078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:41.099975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d25977-a0b7-4124-9a86-f5543782b9d4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:41.196389 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:41.197832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:41.198128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:41.198400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] PATH_INFO: `/v3/projects/cbfb862edafb40d7bd49a7773fd6f45a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:41.199809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfb862edafb40d7bd49a7773fd6f45a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:41.201843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:41.211455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:41.258574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfb862edafb40d7bd49a7773fd6f45a', 'request_id': 'req-df35aaac-d6e4-4c58-b89a-25db09f0f990'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:41.274624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfb862edafb40d7bd49a7773fd6f45a', 'request_id': 'req-df35aaac-d6e4-4c58-b89a-25db09f0f990'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:41.288287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfb862edafb40d7bd49a7773fd6f45a', 'request_id': 'req-df35aaac-d6e4-4c58-b89a-25db09f0f990'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:41.313745 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] The token cache is being invalidate because project cbfb862edafb40d7bd49a7773fd6f45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:41.314177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:41.315129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-df35aaac-d6e4-4c58-b89a-25db09f0f990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:41.317143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1245/2451] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:41 2022] DELETE /identity/v3/projects/cbfb862edafb40d7bd49a7773fd6f45a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:29:41.325242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491051d-6245-4128-992c-708308616e4e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:41.325782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491051d-6245-4128-992c-708308616e4e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:41.326281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491051d-6245-4128-992c-708308616e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:41.349824 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b491051d-6245-4128-992c-708308616e4e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:41.351068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b491051d-6245-4128-992c-708308616e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:41.427001 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1207/2452] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:41.436832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f22f-29b9-45b8-8f32-c95529ffcf01 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:41.437349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f22f-29b9-45b8-8f32-c95529ffcf01 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:41.437901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f22f-29b9-45b8-8f32-c95529ffcf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:41.464721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8a92f22f-29b9-45b8-8f32-c95529ffcf01 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:41.466186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8a92f22f-29b9-45b8-8f32-c95529ffcf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:41.475231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1246/2453] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:29:41.481054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491051d-6245-4128-992c-708308616e4e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:41.492016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491051d-6245-4128-992c-708308616e4e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:41.497201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d18f-d5f6-46a9-8a40-ba1a21b4344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:41.498022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d18f-d5f6-46a9-8a40-ba1a21b4344c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:41.498307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d18f-d5f6-46a9-8a40-ba1a21b4344c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:41.498585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d18f-d5f6-46a9-8a40-ba1a21b4344c admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:41.499367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d18f-d5f6-46a9-8a40-ba1a21b4344c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:29:41.499673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d18f-d5f6-46a9-8a40-ba1a21b4344c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:41.501405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d18f-d5f6-46a9-8a40-ba1a21b4344c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:41.517357 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1208/2454] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:29:41.523783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92f22f-29b9-45b8-8f32-c95529ffcf01 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:41.535105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92f22f-29b9-45b8-8f32-c95529ffcf01 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:41.540059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6040d-5aa8-403a-b35e-fc4fe5fe27d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:41.541409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6040d-5aa8-403a-b35e-fc4fe5fe27d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:41.541696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6040d-5aa8-403a-b35e-fc4fe5fe27d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:41.542009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6040d-5aa8-403a-b35e-fc4fe5fe27d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:41.542730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6040d-5aa8-403a-b35e-fc4fe5fe27d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:41.544701 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6040d-5aa8-403a-b35e-fc4fe5fe27d0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:41.589942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1247/2455] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:41 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:41.597627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d18f-d5f6-46a9-8a40-ba1a21b4344c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:41.642585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d18f-d5f6-46a9-8a40-ba1a21b4344c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:41.702349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:41.768259 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b05162-2604-4b95-a3d2-295c43913834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:41.769298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b05162-2604-4b95-a3d2-295c43913834 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:41.769588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b05162-2604-4b95-a3d2-295c43913834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:41.769924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b05162-2604-4b95-a3d2-295c43913834 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:41.770744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b05162-2604-4b95-a3d2-295c43913834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:41.772787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b05162-2604-4b95-a3d2-295c43913834 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:41.799670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1209/2456] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:41 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:41.805972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6040d-5aa8-403a-b35e-fc4fe5fe27d0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:41.821478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6040d-5aa8-403a-b35e-fc4fe5fe27d0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:41.829973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ed0b4-5794-4523-b100-7422d8656496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:41.831052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ed0b4-5794-4523-b100-7422d8656496 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:41.831529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ed0b4-5794-4523-b100-7422d8656496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:41.831964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ed0b4-5794-4523-b100-7422d8656496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:41.833292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ed0b4-5794-4523-b100-7422d8656496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:41.836156 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ed0b4-5794-4523-b100-7422d8656496 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:41.857036 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1248/2457] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:29:41.858956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b05162-2604-4b95-a3d2-295c43913834 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:41.876174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b05162-2604-4b95-a3d2-295c43913834 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:41.892075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1dc3f-2928-4189-9c7a-22da836940c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:41.892986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1dc3f-2928-4189-9c7a-22da836940c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:41.893528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1dc3f-2928-4189-9c7a-22da836940c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:41.894027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1dc3f-2928-4189-9c7a-22da836940c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:41.895165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1dc3f-2928-4189-9c7a-22da836940c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:41.897288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1dc3f-2928-4189-9c7a-22da836940c6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:41.917613 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1210/2458] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:41.923059 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ed0b4-5794-4523-b100-7422d8656496 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:41.941139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ed0b4-5794-4523-b100-7422d8656496 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:41.949748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48b257-a65b-4504-b69f-18f80b2b389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:41.951349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48b257-a65b-4504-b69f-18f80b2b389d admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:41.951949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48b257-a65b-4504-b69f-18f80b2b389d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:41.952681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48b257-a65b-4504-b69f-18f80b2b389d admin admin] PATH_INFO: `/v3/projects/a5675127501346628ad30214f755307c/users/091aaf78dfc84e24aac996790e601bd8/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:41.980641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48b257-a65b-4504-b69f-18f80b2b389d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5675127501346628ad30214f755307c, user_id=091aaf78dfc84e24aac996790e601bd8, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:41.985687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48b257-a65b-4504-b69f-18f80b2b389d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:42.024286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1249/2459] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:41 2022] PUT /identity/v3/projects/a5675127501346628ad30214f755307c/users/091aaf78dfc84e24aac996790e601bd8/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:29:42.046364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904da4f-3f12-409f-bdc5-10ea016de907 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:42.046868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904da4f-3f12-409f-bdc5-10ea016de907 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:42.047366 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904da4f-3f12-409f-bdc5-10ea016de907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:42.114704 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b904da4f-3f12-409f-bdc5-10ea016de907 None None] MFA Rules not processed for user `091aaf78dfc84e24aac996790e601bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:42.115946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b904da4f-3f12-409f-bdc5-10ea016de907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:42.225847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1211/2460] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:42.238744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48b257-a65b-4504-b69f-18f80b2b389d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:42.241061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6f48b257-a65b-4504-b69f-18f80b2b389d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:42.337233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48b257-a65b-4504-b69f-18f80b2b389d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:42.342743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0145c-8166-4600-bb5d-c1b2e3e1d6bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:42.343586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0145c-8166-4600-bb5d-c1b2e3e1d6bd service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:42.344057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0145c-8166-4600-bb5d-c1b2e3e1d6bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:42.344434 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0145c-8166-4600-bb5d-c1b2e3e1d6bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:42.356164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0145c-8166-4600-bb5d-c1b2e3e1d6bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:42.357774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0145c-8166-4600-bb5d-c1b2e3e1d6bd service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:42.367940 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1250/2461] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:29:42.403163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904da4f-3f12-409f-bdc5-10ea016de907 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:42.498367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904da4f-3f12-409f-bdc5-10ea016de907 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:42.506618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:42.508116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:42.508567 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:42.509009 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] PATH_INFO: `/v3/users/091aaf78dfc84e24aac996790e601bd8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:42.512067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=091aaf78dfc84e24aac996790e601bd8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:42.521899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:42.585681 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:42.590352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '091aaf78dfc84e24aac996790e601bd8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:42.601434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '091aaf78dfc84e24aac996790e601bd8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:42.602430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '091aaf78dfc84e24aac996790e601bd8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:42.622140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '091aaf78dfc84e24aac996790e601bd8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:42.622890 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '091aaf78dfc84e24aac996790e601bd8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:42.640828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1212/2462] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:29:42 2022] DELETE /identity/v3/users/091aaf78dfc84e24aac996790e601bd8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:29:42.645875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58af6c9-b333-4188-a378-cba803cdc1e0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:42.646166 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58af6c9-b333-4188-a378-cba803cdc1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:42.646448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58af6c9-b333-4188-a378-cba803cdc1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:42.665279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a58af6c9-b333-4188-a378-cba803cdc1e0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:42.667671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a58af6c9-b333-4188-a378-cba803cdc1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:42.704886 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:42.743696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1251/2463] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:42.755742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:42.757474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:42.828228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17ecb0-aa01-48e0-80da-f9d97a908088 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:42.832011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21d010-05f0-4263-8260-0aec7183511a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:42.832674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d010-05f0-4263-8260-0aec7183511a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:42.832957 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d010-05f0-4263-8260-0aec7183511a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:42.833234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d010-05f0-4263-8260-0aec7183511a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:42.844556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21d010-05f0-4263-8260-0aec7183511a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:42.846050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21d010-05f0-4263-8260-0aec7183511a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:42.857909 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1213/2464] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:43.230879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58af6c9-b333-4188-a378-cba803cdc1e0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:43.232370 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a58af6c9-b333-4188-a378-cba803cdc1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:43.245899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58af6c9-b333-4188-a378-cba803cdc1e0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:43.253451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:43.254689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.255124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.255509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] PATH_INFO: `/v3/projects/a5675127501346628ad30214f755307c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.257308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5675127501346628ad30214f755307c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:43.260553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:43.321232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5675127501346628ad30214f755307c', 'request_id': 'req-bead9416-7f80-46b3-b7da-5cd75dc0e6da'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:43.329942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5675127501346628ad30214f755307c', 'request_id': 'req-bead9416-7f80-46b3-b7da-5cd75dc0e6da'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:43.337573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5675127501346628ad30214f755307c', 'request_id': 'req-bead9416-7f80-46b3-b7da-5cd75dc0e6da'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:43.348657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] The token cache is being invalidate because project a5675127501346628ad30214f755307c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:29:43.348955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:43.349644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bead9416-7f80-46b3-b7da-5cd75dc0e6da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:29:43.351245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1252/2465] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:29:43 2022] DELETE /identity/v3/projects/a5675127501346628ad30214f755307c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:43.357533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c31fa8-94ff-40f7-a4f2-48cc1028f06e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.358144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c31fa8-94ff-40f7-a4f2-48cc1028f06e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.359260 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c31fa8-94ff-40f7-a4f2-48cc1028f06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.379561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f4c31fa8-94ff-40f7-a4f2-48cc1028f06e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:43.380786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f4c31fa8-94ff-40f7-a4f2-48cc1028f06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:43.443433 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1214/2466] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:43.451873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0dca-e2fe-471d-897f-85e33b01c8cf None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.452193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0dca-e2fe-471d-897f-85e33b01c8cf None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.452488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0dca-e2fe-471d-897f-85e33b01c8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.472594 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-522b0dca-e2fe-471d-897f-85e33b01c8cf None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:43.473911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-522b0dca-e2fe-471d-897f-85e33b01c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:43.484241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1253/2467] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:43.488672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c31fa8-94ff-40f7-a4f2-48cc1028f06e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:43.500386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c31fa8-94ff-40f7-a4f2-48cc1028f06e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:43.510955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7c5c5-fe12-40e8-b98a-03b896348f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:43.511184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7c5c5-fe12-40e8-b98a-03b896348f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.511438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7c5c5-fe12-40e8-b98a-03b896348f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.511710 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7c5c5-fe12-40e8-b98a-03b896348f4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.512447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7c5c5-fe12-40e8-b98a-03b896348f4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:29:43.512785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7c5c5-fe12-40e8-b98a-03b896348f4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:43.514528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7c5c5-fe12-40e8-b98a-03b896348f4e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:43.528380 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1215/2468] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:29:43.533073 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0dca-e2fe-471d-897f-85e33b01c8cf None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:43.543788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0dca-e2fe-471d-897f-85e33b01c8cf None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:43.550572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95774b19-9292-4835-ae92-697816e7c798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:43.551260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774b19-9292-4835-ae92-697816e7c798 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.551544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774b19-9292-4835-ae92-697816e7c798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.551823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774b19-9292-4835-ae92-697816e7c798 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.552539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95774b19-9292-4835-ae92-697816e7c798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:43.554517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95774b19-9292-4835-ae92-697816e7c798 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:43.590941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1254/2469] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:43 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:43.592627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7c5c5-fe12-40e8-b98a-03b896348f4e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:43.610144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7c5c5-fe12-40e8-b98a-03b896348f4e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:43.681302 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:43.700334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487b5cc-0038-4fdf-b5f5-436e0f42dc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:43.701504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487b5cc-0038-4fdf-b5f5-436e0f42dc32 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.702066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487b5cc-0038-4fdf-b5f5-436e0f42dc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.702583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487b5cc-0038-4fdf-b5f5-436e0f42dc32 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.704434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487b5cc-0038-4fdf-b5f5-436e0f42dc32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:43.707818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487b5cc-0038-4fdf-b5f5-436e0f42dc32 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:43.738349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1216/2470] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:43 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:43.742410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95774b19-9292-4835-ae92-697816e7c798 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:43.753422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95774b19-9292-4835-ae92-697816e7c798 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:43.758817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70576427-8c58-4331-8eed-e65ef2de1963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:43.759544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70576427-8c58-4331-8eed-e65ef2de1963 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.760037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70576427-8c58-4331-8eed-e65ef2de1963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.760459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70576427-8c58-4331-8eed-e65ef2de1963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.761202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70576427-8c58-4331-8eed-e65ef2de1963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:43.762885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70576427-8c58-4331-8eed-e65ef2de1963 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:43.779122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1255/2471] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:43.782677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487b5cc-0038-4fdf-b5f5-436e0f42dc32 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:43.795514 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487b5cc-0038-4fdf-b5f5-436e0f42dc32 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:43.801059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2a385-d587-408f-87a6-79b55987f0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:43.801959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2a385-d587-408f-87a6-79b55987f0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.802250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2a385-d587-408f-87a6-79b55987f0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.802633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2a385-d587-408f-87a6-79b55987f0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.803485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2a385-d587-408f-87a6-79b55987f0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:43.805101 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2a385-d587-408f-87a6-79b55987f0e0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:43.816232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1217/2472] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:29:43.822247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70576427-8c58-4331-8eed-e65ef2de1963 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:43.833532 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70576427-8c58-4331-8eed-e65ef2de1963 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:43.838870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67d5ef-acdc-48ad-b7b9-0d3da622adbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:43.839883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67d5ef-acdc-48ad-b7b9-0d3da622adbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.840169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67d5ef-acdc-48ad-b7b9-0d3da622adbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.840442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67d5ef-acdc-48ad-b7b9-0d3da622adbe admin admin] PATH_INFO: `/v3/projects/add30fb78c80414599ebb8f57fb31389/users/4e867646e7734a24a574fcbce0b17c8b/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.862909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67d5ef-acdc-48ad-b7b9-0d3da622adbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=add30fb78c80414599ebb8f57fb31389, user_id=4e867646e7734a24a574fcbce0b17c8b, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:43.867705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67d5ef-acdc-48ad-b7b9-0d3da622adbe admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:43.897280 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1256/2473] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:43 2022] PUT /identity/v3/projects/add30fb78c80414599ebb8f57fb31389/users/4e867646e7734a24a574fcbce0b17c8b/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:43.905672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a3338-75ce-4354-a05d-e2c10e60c521 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:43.906072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a3338-75ce-4354-a05d-e2c10e60c521 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:43.906354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a3338-75ce-4354-a05d-e2c10e60c521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:43.934951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e67a3338-75ce-4354-a05d-e2c10e60c521 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:43.936754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e67a3338-75ce-4354-a05d-e2c10e60c521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:44.018527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1218/2474] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:44.029419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67d5ef-acdc-48ad-b7b9-0d3da622adbe admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:44.031154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4d67d5ef-acdc-48ad-b7b9-0d3da622adbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:44.108144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67d5ef-acdc-48ad-b7b9-0d3da622adbe admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:44.112196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7591a-94bb-4fa5-8673-815add385a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:44.112868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7591a-94bb-4fa5-8673-815add385a40 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:44.113152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7591a-94bb-4fa5-8673-815add385a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:44.113431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7591a-94bb-4fa5-8673-815add385a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:44.123651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7591a-94bb-4fa5-8673-815add385a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:44.125140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7591a-94bb-4fa5-8673-815add385a40 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:44.129895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:44.142314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1257/2475] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:29:53.157833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d97c18-1747-4903-8d4d-b4e94d3dc5b1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:53.158351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d97c18-1747-4903-8d4d-b4e94d3dc5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:53.158450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d97c18-1747-4903-8d4d-b4e94d3dc5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:53.203144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-27d97c18-1747-4903-8d4d-b4e94d3dc5b1 None None] MFA Rules not processed for user `4e867646e7734a24a574fcbce0b17c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:29:53.204363 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-27d97c18-1747-4903-8d4d-b4e94d3dc5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:53.246641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7591a-94bb-4fa5-8673-815add385a40 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:53.249167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a9c7591a-94bb-4fa5-8673-815add385a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:53.285333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7591a-94bb-4fa5-8673-815add385a40 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:53.292869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21939fd9-f45c-4199-8e3a-d22a940c06e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:53.294272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21939fd9-f45c-4199-8e3a-d22a940c06e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:53.295304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21939fd9-f45c-4199-8e3a-d22a940c06e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:53.296002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21939fd9-f45c-4199-8e3a-d22a940c06e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:53.307752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-21939fd9-f45c-4199-8e3a-d22a940c06e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:53.332527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21939fd9-f45c-4199-8e3a-d22a940c06e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:53.334779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21939fd9-f45c-4199-8e3a-d22a940c06e1 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:53.350134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1258/2476] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:29:53.371416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1219/2477] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:29:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:29:53.377620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21939fd9-f45c-4199-8e3a-d22a940c06e1 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:53.389942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21939fd9-f45c-4199-8e3a-d22a940c06e1 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:53.398903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc0a5f-2882-4ca8-85fe-7eed5ba78474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:53.401068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0a5f-2882-4ca8-85fe-7eed5ba78474 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:53.401884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0a5f-2882-4ca8-85fe-7eed5ba78474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:53.402787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0a5f-2882-4ca8-85fe-7eed5ba78474 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:53.404321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc0a5f-2882-4ca8-85fe-7eed5ba78474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:53.408091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc0a5f-2882-4ca8-85fe-7eed5ba78474 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:53.480228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1259/2478] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:29:53 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:53.483465 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d97c18-1747-4903-8d4d-b4e94d3dc5b1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:53.516825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d97c18-1747-4903-8d4d-b4e94d3dc5b1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:53.647470 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c61e9-9881-4359-8e5f-b8414a1803ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:53.648684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c61e9-9881-4359-8e5f-b8414a1803ab admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:53.649120 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c61e9-9881-4359-8e5f-b8414a1803ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:53.649638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c61e9-9881-4359-8e5f-b8414a1803ab admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:53.651647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c61e9-9881-4359-8e5f-b8414a1803ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:53.655297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c61e9-9881-4359-8e5f-b8414a1803ab admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:53.666498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:53.697372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1220/2479] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:29:53 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:29:53.701470 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc0a5f-2882-4ca8-85fe-7eed5ba78474 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:53.714904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc0a5f-2882-4ca8-85fe-7eed5ba78474 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:53.720439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728622e8-75eb-4ccb-9039-dedf384b33f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:53.721136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728622e8-75eb-4ccb-9039-dedf384b33f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:53.721427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728622e8-75eb-4ccb-9039-dedf384b33f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:53.721738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728622e8-75eb-4ccb-9039-dedf384b33f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:53.722929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728622e8-75eb-4ccb-9039-dedf384b33f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:53.726078 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728622e8-75eb-4ccb-9039-dedf384b33f6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:53.741465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1260/2480] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:53.746114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c61e9-9881-4359-8e5f-b8414a1803ab admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:53.757277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c61e9-9881-4359-8e5f-b8414a1803ab admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:53.763867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aedfd4-6000-4d49-b55e-40820f7d635d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:53.764930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aedfd4-6000-4d49-b55e-40820f7d635d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:53.765211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aedfd4-6000-4d49-b55e-40820f7d635d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:53.765484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aedfd4-6000-4d49-b55e-40820f7d635d admin admin] PATH_INFO: `/v3/projects/046d374b343c49deaa23fb1fbdb85cbe/users/6c237ed331704ae2a43fdecfe860c13e/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:53.789374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aedfd4-6000-4d49-b55e-40820f7d635d admin admin] RBAC: Authorizing `identity:create_grant(project_id=046d374b343c49deaa23fb1fbdb85cbe, user_id=6c237ed331704ae2a43fdecfe860c13e, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:53.792850 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aedfd4-6000-4d49-b55e-40820f7d635d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:53.819024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1221/2481] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:53 2022] PUT /identity/v3/projects/046d374b343c49deaa23fb1fbdb85cbe/users/6c237ed331704ae2a43fdecfe860c13e/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:29:53.827024 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728622e8-75eb-4ccb-9039-dedf384b33f6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:53.842350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728622e8-75eb-4ccb-9039-dedf384b33f6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:53.945502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35d211-c828-44b1-ace2-aa614e987313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:53.947602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35d211-c828-44b1-ace2-aa614e987313 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:53.948045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35d211-c828-44b1-ace2-aa614e987313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:53.948468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35d211-c828-44b1-ace2-aa614e987313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:53.949649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35d211-c828-44b1-ace2-aa614e987313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:53.951625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35d211-c828-44b1-ace2-aa614e987313 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:53.966575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1261/2482] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:29:53.970935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aedfd4-6000-4d49-b55e-40820f7d635d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:53.985917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aedfd4-6000-4d49-b55e-40820f7d635d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:53.992740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b116a-5649-4b29-ae13-9e727bf5d6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:53.994508 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b116a-5649-4b29-ae13-9e727bf5d6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:53.995706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b116a-5649-4b29-ae13-9e727bf5d6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:53.995706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b116a-5649-4b29-ae13-9e727bf5d6ad admin admin] PATH_INFO: `/v3/projects/046d374b343c49deaa23fb1fbdb85cbe/users/6c237ed331704ae2a43fdecfe860c13e/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:53.999401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b116a-5649-4b29-ae13-9e727bf5d6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=046d374b343c49deaa23fb1fbdb85cbe, user_id=6c237ed331704ae2a43fdecfe860c13e, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:54.002180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b116a-5649-4b29-ae13-9e727bf5d6ad admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:54.019126 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1222/2483] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:29:53 2022] PUT /identity/v3/projects/046d374b343c49deaa23fb1fbdb85cbe/users/6c237ed331704ae2a43fdecfe860c13e/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:29:57.472144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35d211-c828-44b1-ace2-aa614e987313 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:29:57.684566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35d211-c828-44b1-ace2-aa614e987313 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:29:57.883028 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ab4fd-b9db-43db-92f3-27726da58bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:29:57.883834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ab4fd-b9db-43db-92f3-27726da58bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:29:57.884197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ab4fd-b9db-43db-92f3-27726da58bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:29:57.884482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ab4fd-b9db-43db-92f3-27726da58bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:29:57.887962 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-341ab4fd-b9db-43db-92f3-27726da58bb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:29:57.930486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:29:58.046935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ab4fd-b9db-43db-92f3-27726da58bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:29:58.051158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ab4fd-b9db-43db-92f3-27726da58bb9 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:29:58.071275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1262/2484] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:01.135962 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b116a-5649-4b29-ae13-9e727bf5d6ad admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:01.137833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d23b116a-5649-4b29-ae13-9e727bf5d6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:01.177516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b116a-5649-4b29-ae13-9e727bf5d6ad admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:01.204976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:01.208089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:01.208777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:01.209505 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] PATH_INFO: `/v3/users/53505ac8ab6142a6bf0c3d6fe1cc721f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:01.216949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53505ac8ab6142a6bf0c3d6fe1cc721f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:01.224148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:01.398583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53505ac8ab6142a6bf0c3d6fe1cc721f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:01.420372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53505ac8ab6142a6bf0c3d6fe1cc721f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:01.421696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53505ac8ab6142a6bf0c3d6fe1cc721f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:01.471921 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53505ac8ab6142a6bf0c3d6fe1cc721f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:01.473406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ddd9bb0c-e49f-43dc-b021-2a5267d55531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53505ac8ab6142a6bf0c3d6fe1cc721f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:01.525740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1223/2485] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:01 2022] DELETE /identity/v3/users/53505ac8ab6142a6bf0c3d6fe1cc721f => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:02.518832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ab4fd-b9db-43db-92f3-27726da58bb9 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:02.541273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ab4fd-b9db-43db-92f3-27726da58bb9 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:02.705236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:02.708316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:02.708861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:02.709606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] PATH_INFO: `/v3/projects/f185078fd436428e818d6dba472bacb8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:02.716764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f185078fd436428e818d6dba472bacb8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:02.722956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:02.887740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f185078fd436428e818d6dba472bacb8', 'request_id': 'req-74cae203-8b3e-49ba-b73e-c9e33401a704'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:02.920957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f185078fd436428e818d6dba472bacb8', 'request_id': 'req-74cae203-8b3e-49ba-b73e-c9e33401a704'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:02.932464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f185078fd436428e818d6dba472bacb8', 'request_id': 'req-74cae203-8b3e-49ba-b73e-c9e33401a704'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:02.949386 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] The token cache is being invalidate because project f185078fd436428e818d6dba472bacb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:02.950088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:02.951510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-74cae203-8b3e-49ba-b73e-c9e33401a704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:02.960553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593b0b6-e183-4d33-9f54-f5db0c4647c7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:02.963308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593b0b6-e183-4d33-9f54-f5db0c4647c7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:02.963877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593b0b6-e183-4d33-9f54-f5db0c4647c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:02.965985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1263/2486] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:02 2022] DELETE /identity/v3/projects/f185078fd436428e818d6dba472bacb8 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:02.992796 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1593b0b6-e183-4d33-9f54-f5db0c4647c7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:02.995785 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1593b0b6-e183-4d33-9f54-f5db0c4647c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:03.123713 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1224/2487] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:03.132696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d81a4-15c8-45f3-8108-3c4128055df7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.139644 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d81a4-15c8-45f3-8108-3c4128055df7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.140262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d81a4-15c8-45f3-8108-3c4128055df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.205562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-698d81a4-15c8-45f3-8108-3c4128055df7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:03.207058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-698d81a4-15c8-45f3-8108-3c4128055df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:03.224878 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00744e78-344f-4f22-b414-8f72509d3362 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.225146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00744e78-344f-4f22-b414-8f72509d3362 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.225439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00744e78-344f-4f22-b414-8f72509d3362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.237698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1264/2488] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:03.247847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d81a4-15c8-45f3-8108-3c4128055df7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:03.268822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d81a4-15c8-45f3-8108-3c4128055df7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:03.286977 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f33e02-44dd-4f7b-9af5-dd6670c3f677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:03.288348 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f33e02-44dd-4f7b-9af5-dd6670c3f677 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.291554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f33e02-44dd-4f7b-9af5-dd6670c3f677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.293091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f33e02-44dd-4f7b-9af5-dd6670c3f677 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.295064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f33e02-44dd-4f7b-9af5-dd6670c3f677 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:03.298673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f33e02-44dd-4f7b-9af5-dd6670c3f677 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:03.300499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f33e02-44dd-4f7b-9af5-dd6670c3f677 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:03.325693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-00744e78-344f-4f22-b414-8f72509d3362 None None] MFA Rules not processed for user `6c237ed331704ae2a43fdecfe860c13e`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:03.325693 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-00744e78-344f-4f22-b414-8f72509d3362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:03.333322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1265/2489] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:03.339473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f33e02-44dd-4f7b-9af5-dd6670c3f677 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:03.364195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f33e02-44dd-4f7b-9af5-dd6670c3f677 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:03.365825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:INSERT=6,keystone:SELECT=84,keystone:DELETE=10 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:03.379505 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b51c0-df68-48ef-ada7-bf7424c5d25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:03.380440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b51c0-df68-48ef-ada7-bf7424c5d25d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.380849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b51c0-df68-48ef-ada7-bf7424c5d25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.381312 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b51c0-df68-48ef-ada7-bf7424c5d25d admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.382209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b51c0-df68-48ef-ada7-bf7424c5d25d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:03.385827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b51c0-df68-48ef-ada7-bf7424c5d25d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:03.457523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1266/2490] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:03 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:03.461906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b51c0-df68-48ef-ada7-bf7424c5d25d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:03.470188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:03.521603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1225/2491] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:03.544047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00744e78-344f-4f22-b414-8f72509d3362 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:03.550706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-00744e78-344f-4f22-b414-8f72509d3362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:03.584947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b51c0-df68-48ef-ada7-bf7424c5d25d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:03.594878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3013-ab21-4d1e-8d78-33542d644955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:03.596029 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3013-ab21-4d1e-8d78-33542d644955 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.596520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3013-ab21-4d1e-8d78-33542d644955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.597012 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3013-ab21-4d1e-8d78-33542d644955 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.598557 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f3013-ab21-4d1e-8d78-33542d644955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:03.603801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f3013-ab21-4d1e-8d78-33542d644955 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:03.650857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1267/2492] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:03 2022] POST /identity/v3/users => generated 283 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:03.654737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3013-ab21-4d1e-8d78-33542d644955 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:03.668372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3013-ab21-4d1e-8d78-33542d644955 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:03.674871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e979-ef65-4082-bcde-d77720a640f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:03.675718 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e979-ef65-4082-bcde-d77720a640f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.676010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e979-ef65-4082-bcde-d77720a640f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.676289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e979-ef65-4082-bcde-d77720a640f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.677257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189e979-ef65-4082-bcde-d77720a640f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:03.679531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189e979-ef65-4082-bcde-d77720a640f8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:03.703892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1268/2493] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:03.705157 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e979-ef65-4082-bcde-d77720a640f8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:03.721744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e979-ef65-4082-bcde-d77720a640f8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:03.728917 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b1dfd-ced8-4273-b041-373778951d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:03.730257 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b1dfd-ced8-4273-b041-373778951d07 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.730765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b1dfd-ced8-4273-b041-373778951d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.731066 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b1dfd-ced8-4273-b041-373778951d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.731958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b1dfd-ced8-4273-b041-373778951d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:03.734030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b1dfd-ced8-4273-b041-373778951d07 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:03.758446 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1269/2494] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:03.759801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00744e78-344f-4f22-b414-8f72509d3362 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:03.761445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b1dfd-ced8-4273-b041-373778951d07 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:03.777072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b1dfd-ced8-4273-b041-373778951d07 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:03.785677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475408c-ca1f-408a-9597-ef9e65d71671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:03.787380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475408c-ca1f-408a-9597-ef9e65d71671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.787884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475408c-ca1f-408a-9597-ef9e65d71671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.788375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475408c-ca1f-408a-9597-ef9e65d71671 admin admin] PATH_INFO: `/v3/projects/fbea9422f6fe4521a49803cfe7e43e4b/users/3860c7cd45574d3a9c6eddfaa9697a7f/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.823168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475408c-ca1f-408a-9597-ef9e65d71671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbea9422f6fe4521a49803cfe7e43e4b, user_id=3860c7cd45574d3a9c6eddfaa9697a7f, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:03.827621 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475408c-ca1f-408a-9597-ef9e65d71671 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:03.861195 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1270/2495] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:03 2022] PUT /identity/v3/projects/fbea9422f6fe4521a49803cfe7e43e4b/users/3860c7cd45574d3a9c6eddfaa9697a7f/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:03.874278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6420eb8-65f8-42fc-bcd7-e28e34536990 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.874819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6420eb8-65f8-42fc-bcd7-e28e34536990 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.875668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6420eb8-65f8-42fc-bcd7-e28e34536990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.916950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e11915-712c-4087-b6fa-5523b6265fc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:03.919267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e11915-712c-4087-b6fa-5523b6265fc9 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:03.920102 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e11915-712c-4087-b6fa-5523b6265fc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:03.920983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e11915-712c-4087-b6fa-5523b6265fc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:03.925187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a5e11915-712c-4087-b6fa-5523b6265fc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:03.951863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a6420eb8-65f8-42fc-bcd7-e28e34536990 None None] MFA Rules not processed for user `3860c7cd45574d3a9c6eddfaa9697a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:03.955313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a6420eb8-65f8-42fc-bcd7-e28e34536990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:04.017153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:04.048634 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e11915-712c-4087-b6fa-5523b6265fc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:04.051008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e11915-712c-4087-b6fa-5523b6265fc9 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:04.070462 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1226/2496] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:04.110431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1271/2497] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:04.117623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e11915-712c-4087-b6fa-5523b6265fc9 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:04.142325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e11915-712c-4087-b6fa-5523b6265fc9 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:04.238232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784362b1-b869-440f-a2a2-74a910efaad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:04.240743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784362b1-b869-440f-a2a2-74a910efaad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:04.241432 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784362b1-b869-440f-a2a2-74a910efaad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:04.242472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784362b1-b869-440f-a2a2-74a910efaad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:04.244072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784362b1-b869-440f-a2a2-74a910efaad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:04.247638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784362b1-b869-440f-a2a2-74a910efaad5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:04.311896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1227/2498] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:04 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:04.319772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6420eb8-65f8-42fc-bcd7-e28e34536990 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:04.350562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6420eb8-65f8-42fc-bcd7-e28e34536990 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:04.420274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:04.488190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8eba-66eb-4fe2-b81a-9761bb7a23cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:04.489398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8eba-66eb-4fe2-b81a-9761bb7a23cb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:04.490138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8eba-66eb-4fe2-b81a-9761bb7a23cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:04.491517 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8eba-66eb-4fe2-b81a-9761bb7a23cb admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:04.494937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8eba-66eb-4fe2-b81a-9761bb7a23cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:04.511087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8eba-66eb-4fe2-b81a-9761bb7a23cb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:04.576091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1272/2499] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:04 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:04.577105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784362b1-b869-440f-a2a2-74a910efaad5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:04.595984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784362b1-b869-440f-a2a2-74a910efaad5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:04.601638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab116247-c723-482b-ab06-3ecc7ac32241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:04.605589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab116247-c723-482b-ab06-3ecc7ac32241 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:04.606000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab116247-c723-482b-ab06-3ecc7ac32241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:04.606439 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab116247-c723-482b-ab06-3ecc7ac32241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:04.607307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab116247-c723-482b-ab06-3ecc7ac32241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:04.612118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab116247-c723-482b-ab06-3ecc7ac32241 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:04.637951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1228/2500] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:04.638260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8eba-66eb-4fe2-b81a-9761bb7a23cb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:04.654652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8eba-66eb-4fe2-b81a-9761bb7a23cb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:04.671778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9aaa6-0e65-4e4c-94ca-0a10b61f8969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:04.673186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9aaa6-0e65-4e4c-94ca-0a10b61f8969 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:04.673669 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9aaa6-0e65-4e4c-94ca-0a10b61f8969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:04.674951 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9aaa6-0e65-4e4c-94ca-0a10b61f8969 admin admin] PATH_INFO: `/v3/projects/e677549e4c1146e28d89ce4e0f9d6ec1/users/e3dca51a476342688db661374ae5800c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:04.715359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9aaa6-0e65-4e4c-94ca-0a10b61f8969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e677549e4c1146e28d89ce4e0f9d6ec1, user_id=e3dca51a476342688db661374ae5800c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:04.722695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9aaa6-0e65-4e4c-94ca-0a10b61f8969 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:04.748737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1273/2501] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:04 2022] PUT /identity/v3/projects/e677549e4c1146e28d89ce4e0f9d6ec1/users/e3dca51a476342688db661374ae5800c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:04.753041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab116247-c723-482b-ab06-3ecc7ac32241 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:04.777775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab116247-c723-482b-ab06-3ecc7ac32241 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:04.911007 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ba485-4d28-4a7a-a0d6-bc75e0bb705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:04.911833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ba485-4d28-4a7a-a0d6-bc75e0bb705c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:04.913129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ba485-4d28-4a7a-a0d6-bc75e0bb705c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:04.913480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ba485-4d28-4a7a-a0d6-bc75e0bb705c admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:04.914407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ba485-4d28-4a7a-a0d6-bc75e0bb705c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:04.917251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ba485-4d28-4a7a-a0d6-bc75e0bb705c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:04.935442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1229/2502] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:04.940497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9aaa6-0e65-4e4c-94ca-0a10b61f8969 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:04.958817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9aaa6-0e65-4e4c-94ca-0a10b61f8969 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:04.968038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadd637-291d-4d56-a546-4b16746dede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:04.969406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadd637-291d-4d56-a546-4b16746dede4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:04.969801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadd637-291d-4d56-a546-4b16746dede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:04.970180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadd637-291d-4d56-a546-4b16746dede4 admin admin] PATH_INFO: `/v3/projects/e677549e4c1146e28d89ce4e0f9d6ec1/users/e3dca51a476342688db661374ae5800c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:04.972523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eadd637-291d-4d56-a546-4b16746dede4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e677549e4c1146e28d89ce4e0f9d6ec1, user_id=e3dca51a476342688db661374ae5800c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:04.983898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eadd637-291d-4d56-a546-4b16746dede4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:05.003774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1274/2503] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:04 2022] PUT /identity/v3/projects/e677549e4c1146e28d89ce4e0f9d6ec1/users/e3dca51a476342688db661374ae5800c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:05.015442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8c25e-476b-43ec-95fa-7656828ae209 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:05.018447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8c25e-476b-43ec-95fa-7656828ae209 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:05.019806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8c25e-476b-43ec-95fa-7656828ae209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:05.059184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fbb8c25e-476b-43ec-95fa-7656828ae209 None None] MFA Rules not processed for user `e3dca51a476342688db661374ae5800c`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:05.061321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fbb8c25e-476b-43ec-95fa-7656828ae209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:05.150378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:05.229495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1230/2504] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:05.248210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadd637-291d-4d56-a546-4b16746dede4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:05.252786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3eadd637-291d-4d56-a546-4b16746dede4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:05.429339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadd637-291d-4d56-a546-4b16746dede4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:05.445297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae19953-f8db-4f21-b1e8-fe73f975b176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:05.447870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19953-f8db-4f21-b1e8-fe73f975b176 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:05.450266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19953-f8db-4f21-b1e8-fe73f975b176 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:05.450913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19953-f8db-4f21-b1e8-fe73f975b176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:05.485217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae19953-f8db-4f21-b1e8-fe73f975b176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:05.488419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae19953-f8db-4f21-b1e8-fe73f975b176 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:05.519562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1275/2505] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:05.670845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8c25e-476b-43ec-95fa-7656828ae209 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:05.700274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8c25e-476b-43ec-95fa-7656828ae209 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:05.715989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e48138-1146-48bc-bec6-03bee8a3a90f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:05.716743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e48138-1146-48bc-bec6-03bee8a3a90f service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:05.717203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e48138-1146-48bc-bec6-03bee8a3a90f service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:05.717608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e48138-1146-48bc-bec6-03bee8a3a90f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:05.735450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae19953-f8db-4f21-b1e8-fe73f975b176 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:05.738866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cae19953-f8db-4f21-b1e8-fe73f975b176 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:05.857452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e48138-1146-48bc-bec6-03bee8a3a90f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:05.859690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e48138-1146-48bc-bec6-03bee8a3a90f service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:05.880039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1231/2506] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:05.918120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae19953-f8db-4f21-b1e8-fe73f975b176 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:05.931168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:05.936320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:05.936780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:05.937079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] PATH_INFO: `/v3/users/3f2e1c5d022c4bde9e76b8a933739557` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:05.939118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f2e1c5d022c4bde9e76b8a933739557)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:05.941909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:06.051716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:06.087172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f2e1c5d022c4bde9e76b8a933739557'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:06.113740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f2e1c5d022c4bde9e76b8a933739557'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:06.115518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f2e1c5d022c4bde9e76b8a933739557'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:06.140234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f2e1c5d022c4bde9e76b8a933739557'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:06.184493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f2e1c5d022c4bde9e76b8a933739557'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:06.197692 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1276/2507] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:05 2022] DELETE /identity/v3/users/3f2e1c5d022c4bde9e76b8a933739557 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:06.239359 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5739a0-7f8c-4c50-b951-3473c921f65d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:06.240502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5739a0-7f8c-4c50-b951-3473c921f65d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:06.240906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5739a0-7f8c-4c50-b951-3473c921f65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:06.264149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3c5739a0-7f8c-4c50-b951-3473c921f65d None None] MFA Rules not processed for user `4e867646e7734a24a574fcbce0b17c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:06.265730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3c5739a0-7f8c-4c50-b951-3473c921f65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:06.393046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1232/2508] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:06.652310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:06.653830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:06.845839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a10b62-ddd3-46cf-86f4-a65219756ddf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:06.851978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:06.867508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:06.868685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:06.869295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] PATH_INFO: `/v3/projects/c41996fbf4ab4b81bc47ac29252ec762` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:06.874189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c41996fbf4ab4b81bc47ac29252ec762)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:06.881947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:07.039483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c41996fbf4ab4b81bc47ac29252ec762', 'request_id': 'req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:07.065595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c41996fbf4ab4b81bc47ac29252ec762', 'request_id': 'req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:07.081146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c41996fbf4ab4b81bc47ac29252ec762', 'request_id': 'req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:07.112550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] The token cache is being invalidate because project c41996fbf4ab4b81bc47ac29252ec762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:07.112901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:07.114122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41f9e0b6-8d5a-4868-a944-6c2e40147ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:07.118263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1277/2509] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:06 2022] DELETE /identity/v3/projects/c41996fbf4ab4b81bc47ac29252ec762 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:07.124035 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25be7-1d45-4556-86cb-4e422e4811ed None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:07.124556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25be7-1d45-4556-86cb-4e422e4811ed None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:07.124995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25be7-1d45-4556-86cb-4e422e4811ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:07.157682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a6d25be7-1d45-4556-86cb-4e422e4811ed None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:07.159728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a6d25be7-1d45-4556-86cb-4e422e4811ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:07.338392 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1233/2510] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:07.350399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446c2f2-2ace-4953-9333-058434a524d3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:07.351254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446c2f2-2ace-4953-9333-058434a524d3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:07.351874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446c2f2-2ace-4953-9333-058434a524d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:07.394868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2446c2f2-2ace-4953-9333-058434a524d3 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:07.396143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2446c2f2-2ace-4953-9333-058434a524d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:07.417407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1278/2511] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:07.422388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25be7-1d45-4556-86cb-4e422e4811ed None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:07.433508 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25be7-1d45-4556-86cb-4e422e4811ed None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:07.440993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71d470-fab1-4a39-a81d-fb91bc05479a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:07.441965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71d470-fab1-4a39-a81d-fb91bc05479a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:07.442478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71d470-fab1-4a39-a81d-fb91bc05479a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:07.442924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71d470-fab1-4a39-a81d-fb91bc05479a admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:07.443888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71d470-fab1-4a39-a81d-fb91bc05479a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:07.444364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71d470-fab1-4a39-a81d-fb91bc05479a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:07.447418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71d470-fab1-4a39-a81d-fb91bc05479a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:07.466617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1234/2512] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:07.470887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446c2f2-2ace-4953-9333-058434a524d3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:07.482171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446c2f2-2ace-4953-9333-058434a524d3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:07.487927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e7061-84b8-4fa7-b05d-279ace858983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:07.488787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e7061-84b8-4fa7-b05d-279ace858983 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:07.489228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e7061-84b8-4fa7-b05d-279ace858983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:07.489583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e7061-84b8-4fa7-b05d-279ace858983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:07.490640 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e7061-84b8-4fa7-b05d-279ace858983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:07.493576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e7061-84b8-4fa7-b05d-279ace858983 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:07.536863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1279/2513] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:07 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:07.542847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71d470-fab1-4a39-a81d-fb91bc05479a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:07.575630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71d470-fab1-4a39-a81d-fb91bc05479a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:07.594991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:07.774323 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b6830-9a6d-466c-b4ff-3ba873eba3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:07.778901 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b6830-9a6d-466c-b4ff-3ba873eba3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:07.780163 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b6830-9a6d-466c-b4ff-3ba873eba3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:07.780906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b6830-9a6d-466c-b4ff-3ba873eba3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:07.783144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b6830-9a6d-466c-b4ff-3ba873eba3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:07.789686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b6830-9a6d-466c-b4ff-3ba873eba3c0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:07.834927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1235/2514] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:07 2022] POST /identity/v3/users => generated 283 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:07.853266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e7061-84b8-4fa7-b05d-279ace858983 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:07.892159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e7061-84b8-4fa7-b05d-279ace858983 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:07.903451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2405d43-c456-4502-900f-eab13669204a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:07.905116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2405d43-c456-4502-900f-eab13669204a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:07.906014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2405d43-c456-4502-900f-eab13669204a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:07.907004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2405d43-c456-4502-900f-eab13669204a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:07.908976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2405d43-c456-4502-900f-eab13669204a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:07.912241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2405d43-c456-4502-900f-eab13669204a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:07.934003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:07.955248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1280/2515] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:07.955935 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b6830-9a6d-466c-b4ff-3ba873eba3c0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:07.985854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b6830-9a6d-466c-b4ff-3ba873eba3c0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:07.997996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b05e3-b18c-482e-879e-803a8540f24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:07.999628 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b05e3-b18c-482e-879e-803a8540f24e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:08.000831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b05e3-b18c-482e-879e-803a8540f24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:08.001029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b05e3-b18c-482e-879e-803a8540f24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:08.015910 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b05e3-b18c-482e-879e-803a8540f24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:08.024178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b05e3-b18c-482e-879e-803a8540f24e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:08.054244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1236/2516] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:08.059123 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2405d43-c456-4502-900f-eab13669204a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:08.082039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2405d43-c456-4502-900f-eab13669204a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:08.089631 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79deef8c-9d8c-4d8d-8b73-fb0bdfc211bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:08.091379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79deef8c-9d8c-4d8d-8b73-fb0bdfc211bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:08.091980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79deef8c-9d8c-4d8d-8b73-fb0bdfc211bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:08.092547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79deef8c-9d8c-4d8d-8b73-fb0bdfc211bb admin admin] PATH_INFO: `/v3/projects/ad53a7d4fd8c4355b9493e8b8d83b255/users/945bd65306e945588ea773f9e3602865/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:08.129122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79deef8c-9d8c-4d8d-8b73-fb0bdfc211bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad53a7d4fd8c4355b9493e8b8d83b255, user_id=945bd65306e945588ea773f9e3602865, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:08.134859 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79deef8c-9d8c-4d8d-8b73-fb0bdfc211bb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:08.157448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1281/2517] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:08 2022] PUT /identity/v3/projects/ad53a7d4fd8c4355b9493e8b8d83b255/users/945bd65306e945588ea773f9e3602865/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:08.169430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aac084-bdaa-40f2-9b1e-73f435f26a55 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:08.169742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aac084-bdaa-40f2-9b1e-73f435f26a55 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:08.170048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aac084-bdaa-40f2-9b1e-73f435f26a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:08.209993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b9aac084-bdaa-40f2-9b1e-73f435f26a55 None None] MFA Rules not processed for user `945bd65306e945588ea773f9e3602865`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:08.211482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b9aac084-bdaa-40f2-9b1e-73f435f26a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:08.379284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1237/2518] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:08.384504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79deef8c-9d8c-4d8d-8b73-fb0bdfc211bb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:08.462942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79deef8c-9d8c-4d8d-8b73-fb0bdfc211bb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:08.468783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd3568-b6fb-429d-a28e-3306a13a7b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:08.469512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd3568-b6fb-429d-a28e-3306a13a7b3b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:08.469851 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd3568-b6fb-429d-a28e-3306a13a7b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:08.470134 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd3568-b6fb-429d-a28e-3306a13a7b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:08.470861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd3568-b6fb-429d-a28e-3306a13a7b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:08.472963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd3568-b6fb-429d-a28e-3306a13a7b3b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:08.511232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1282/2519] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:08 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:08.516516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aac084-bdaa-40f2-9b1e-73f435f26a55 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:08.546158 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aac084-bdaa-40f2-9b1e-73f435f26a55 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:08.631984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d923c-defe-4952-9eda-abf1afc170ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:08.633277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d923c-defe-4952-9eda-abf1afc170ad admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:08.634252 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d923c-defe-4952-9eda-abf1afc170ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:08.634675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d923c-defe-4952-9eda-abf1afc170ad admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:08.635623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d923c-defe-4952-9eda-abf1afc170ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:08.638997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d923c-defe-4952-9eda-abf1afc170ad admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:08.670135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1238/2520] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:08 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:08.676320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd3568-b6fb-429d-a28e-3306a13a7b3b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:08.689585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd3568-b6fb-429d-a28e-3306a13a7b3b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:08.695063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c38a6c-68d3-44cb-b8a7-96cb177a58da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:08.697650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c38a6c-68d3-44cb-b8a7-96cb177a58da admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:08.699199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c38a6c-68d3-44cb-b8a7-96cb177a58da admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:08.699199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c38a6c-68d3-44cb-b8a7-96cb177a58da admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:08.699758 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c38a6c-68d3-44cb-b8a7-96cb177a58da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:08.702872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c38a6c-68d3-44cb-b8a7-96cb177a58da admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:08.726478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1283/2521] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:08.729326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d923c-defe-4952-9eda-abf1afc170ad admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:08.744737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d923c-defe-4952-9eda-abf1afc170ad admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:08.756181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768cc4-a2a1-4f37-b2d5-e1eea75f13c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:08.757647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768cc4-a2a1-4f37-b2d5-e1eea75f13c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:08.758034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768cc4-a2a1-4f37-b2d5-e1eea75f13c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:08.758955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768cc4-a2a1-4f37-b2d5-e1eea75f13c2 admin admin] PATH_INFO: `/v3/projects/88bc6a79e0ef40ac963c787219f2e85f/users/b2451f5447c249eeb90eaf216f9c83cb/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:08.768080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:08.799039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4768cc4-a2a1-4f37-b2d5-e1eea75f13c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88bc6a79e0ef40ac963c787219f2e85f, user_id=b2451f5447c249eeb90eaf216f9c83cb, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:08.802882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4768cc4-a2a1-4f37-b2d5-e1eea75f13c2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:08.830714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1239/2522] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:08 2022] PUT /identity/v3/projects/88bc6a79e0ef40ac963c787219f2e85f/users/b2451f5447c249eeb90eaf216f9c83cb/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:08.837629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c38a6c-68d3-44cb-b8a7-96cb177a58da admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:08.848809 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c38a6c-68d3-44cb-b8a7-96cb177a58da admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:08.942499 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df1208-3490-4e0a-8dce-fa8dc6f35553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:08.945322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df1208-3490-4e0a-8dce-fa8dc6f35553 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:08.945322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df1208-3490-4e0a-8dce-fa8dc6f35553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:08.945322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df1208-3490-4e0a-8dce-fa8dc6f35553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:08.945322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df1208-3490-4e0a-8dce-fa8dc6f35553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:08.949683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df1208-3490-4e0a-8dce-fa8dc6f35553 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:08.965754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1284/2523] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:08.970764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768cc4-a2a1-4f37-b2d5-e1eea75f13c2 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:08.998787 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768cc4-a2a1-4f37-b2d5-e1eea75f13c2 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:09.006930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6002-a3ab-4a6c-8e3f-2807b3e6b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:09.008121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6002-a3ab-4a6c-8e3f-2807b3e6b624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:09.009114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6002-a3ab-4a6c-8e3f-2807b3e6b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:09.009555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6002-a3ab-4a6c-8e3f-2807b3e6b624 admin admin] PATH_INFO: `/v3/projects/88bc6a79e0ef40ac963c787219f2e85f/users/b2451f5447c249eeb90eaf216f9c83cb/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:09.012297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c6002-a3ab-4a6c-8e3f-2807b3e6b624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88bc6a79e0ef40ac963c787219f2e85f, user_id=b2451f5447c249eeb90eaf216f9c83cb, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:09.015564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c6002-a3ab-4a6c-8e3f-2807b3e6b624 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:09.028731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1240/2524] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:08 2022] PUT /identity/v3/projects/88bc6a79e0ef40ac963c787219f2e85f/users/b2451f5447c249eeb90eaf216f9c83cb/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:09.039137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e1acc-c278-40fb-9a7a-8f51fc597327 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:09.039405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e1acc-c278-40fb-9a7a-8f51fc597327 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:09.039827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e1acc-c278-40fb-9a7a-8f51fc597327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:09.091520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f66e1acc-c278-40fb-9a7a-8f51fc597327 None None] MFA Rules not processed for user `b2451f5447c249eeb90eaf216f9c83cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:09.094381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f66e1acc-c278-40fb-9a7a-8f51fc597327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:09.203854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:09.271687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1285/2525] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:09.301064 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6002-a3ab-4a6c-8e3f-2807b3e6b624 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:09.304489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4e2c6002-a3ab-4a6c-8e3f-2807b3e6b624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:09.417516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6002-a3ab-4a6c-8e3f-2807b3e6b624 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:09.422403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909c6c8-6065-4fa9-9cb1-fd8b6c7627bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:09.423620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909c6c8-6065-4fa9-9cb1-fd8b6c7627bd service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:09.424022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909c6c8-6065-4fa9-9cb1-fd8b6c7627bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:09.424428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909c6c8-6065-4fa9-9cb1-fd8b6c7627bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:09.449039 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909c6c8-6065-4fa9-9cb1-fd8b6c7627bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:09.451891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909c6c8-6065-4fa9-9cb1-fd8b6c7627bd service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:09.465630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1241/2526] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:10.469381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e1acc-c278-40fb-9a7a-8f51fc597327 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:10.604384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e1acc-c278-40fb-9a7a-8f51fc597327 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:10.622591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:10.623663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:10.624048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:10.624427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] PATH_INFO: `/v3/users/945bd65306e945588ea773f9e3602865` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:10.626052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=945bd65306e945588ea773f9e3602865)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:10.628240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:10.705892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945bd65306e945588ea773f9e3602865'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:10.720444 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945bd65306e945588ea773f9e3602865'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:10.721592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945bd65306e945588ea773f9e3602865'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:10.734109 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945bd65306e945588ea773f9e3602865'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:10.762370 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945bd65306e945588ea773f9e3602865'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:10.773824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1286/2527] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:10 2022] DELETE /identity/v3/users/945bd65306e945588ea773f9e3602865 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:10.778822 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d1162-a620-48ed-8071-599962dbcec0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:10.779844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d1162-a620-48ed-8071-599962dbcec0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:10.780387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d1162-a620-48ed-8071-599962dbcec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:10.853117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-120d1162-a620-48ed-8071-599962dbcec0 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:10.857665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-120d1162-a620-48ed-8071-599962dbcec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:10.993441 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1242/2528] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:11.005193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:11.007589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:11.192625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073d20a-9c4a-4ad0-9c62-83c7e71ff53e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:11.202985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120064e-40cc-4559-9836-8a64ab6e030d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:11.204207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120064e-40cc-4559-9836-8a64ab6e030d service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:11.204663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120064e-40cc-4559-9836-8a64ab6e030d service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:11.205140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120064e-40cc-4559-9836-8a64ab6e030d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:11.223095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:11.226304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120064e-40cc-4559-9836-8a64ab6e030d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:11.228333 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120064e-40cc-4559-9836-8a64ab6e030d service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:11.248737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1287/2529] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:30:11.972374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d1162-a620-48ed-8071-599962dbcec0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:11.975458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-120d1162-a620-48ed-8071-599962dbcec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:11.997977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d1162-a620-48ed-8071-599962dbcec0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:12.012992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:12.016654 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:12.016654 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:12.016654 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] PATH_INFO: `/v3/projects/ad53a7d4fd8c4355b9493e8b8d83b255` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:12.019227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad53a7d4fd8c4355b9493e8b8d83b255)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:12.022289 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:12.146204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad53a7d4fd8c4355b9493e8b8d83b255', 'request_id': 'req-84c4159a-53f1-40bf-8655-87d4b54f8577'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.166389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad53a7d4fd8c4355b9493e8b8d83b255', 'request_id': 'req-84c4159a-53f1-40bf-8655-87d4b54f8577'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.194426 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad53a7d4fd8c4355b9493e8b8d83b255', 'request_id': 'req-84c4159a-53f1-40bf-8655-87d4b54f8577'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.213763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:12.231191 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] The token cache is being invalidate because project ad53a7d4fd8c4355b9493e8b8d83b255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:12.232021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.233378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.236798 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1243/2530] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:11 2022] DELETE /identity/v3/projects/ad53a7d4fd8c4355b9493e8b8d83b255 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:12.242438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120064e-40cc-4559-9836-8a64ab6e030d service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:12.258108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120064e-40cc-4559-9836-8a64ab6e030d service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:12.364922 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:12.366147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:12.366523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:12.366894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] PATH_INFO: `/v3/users/b2451f5447c249eeb90eaf216f9c83cb` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:12.369143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2451f5447c249eeb90eaf216f9c83cb)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:12.373789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:12.483899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2451f5447c249eeb90eaf216f9c83cb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.514477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2451f5447c249eeb90eaf216f9c83cb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.515555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2451f5447c249eeb90eaf216f9c83cb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.531687 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2451f5447c249eeb90eaf216f9c83cb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.560494 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ffe9e1c-685d-4a0c-bd10-d6b3ed2ca03f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2451f5447c249eeb90eaf216f9c83cb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:12.585550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1288/2531] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:12 2022] DELETE /identity/v3/users/b2451f5447c249eeb90eaf216f9c83cb => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:13.613982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:13.616552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:13.704516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4159a-53f1-40bf-8655-87d4b54f8577 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:13.867672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:13.870604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:13.871081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:13.871897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] PATH_INFO: `/v3/projects/88bc6a79e0ef40ac963c787219f2e85f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:13.874726 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88bc6a79e0ef40ac963c787219f2e85f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:13.919476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:14.111401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88bc6a79e0ef40ac963c787219f2e85f', 'request_id': 'req-b186939c-e601-45c2-8f46-4989ce28f59c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:14.128374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88bc6a79e0ef40ac963c787219f2e85f', 'request_id': 'req-b186939c-e601-45c2-8f46-4989ce28f59c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:14.136835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88bc6a79e0ef40ac963c787219f2e85f', 'request_id': 'req-b186939c-e601-45c2-8f46-4989ce28f59c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:14.166215 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] The token cache is being invalidate because project 88bc6a79e0ef40ac963c787219f2e85f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:14.166495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:14.168659 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b186939c-e601-45c2-8f46-4989ce28f59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:14.171196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1244/2532] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:13 2022] DELETE /identity/v3/projects/88bc6a79e0ef40ac963c787219f2e85f => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:14.177834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1123f-8189-4884-975d-0b6d6cc3f0c2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:14.178593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1123f-8189-4884-975d-0b6d6cc3f0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:14.178593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1123f-8189-4884-975d-0b6d6cc3f0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:14.232004 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f3d1123f-8189-4884-975d-0b6d6cc3f0c2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:14.235509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f3d1123f-8189-4884-975d-0b6d6cc3f0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:14.409433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1289/2533] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:14.411329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0894e8-757b-4ba0-b90d-caf5b839dcf6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:14.411624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0894e8-757b-4ba0-b90d-caf5b839dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:14.411907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0894e8-757b-4ba0-b90d-caf5b839dcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:14.459677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6b0894e8-757b-4ba0-b90d-caf5b839dcf6 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:14.462876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6b0894e8-757b-4ba0-b90d-caf5b839dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:14.475059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1245/2534] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:14.482705 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1123f-8189-4884-975d-0b6d6cc3f0c2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:14.494902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1123f-8189-4884-975d-0b6d6cc3f0c2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:14.501976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019526ca-9670-4fb4-ae25-15e5efffb9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:14.502716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019526ca-9670-4fb4-ae25-15e5efffb9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:14.503146 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019526ca-9670-4fb4-ae25-15e5efffb9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:14.503496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019526ca-9670-4fb4-ae25-15e5efffb9ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:14.504337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019526ca-9670-4fb4-ae25-15e5efffb9ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:14.504708 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019526ca-9670-4fb4-ae25-15e5efffb9ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:14.506539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019526ca-9670-4fb4-ae25-15e5efffb9ca admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:14.518678 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:14.523855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1290/2535] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:14.528738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0894e8-757b-4ba0-b90d-caf5b839dcf6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:14.538847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0894e8-757b-4ba0-b90d-caf5b839dcf6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:14.545805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45cd76-1925-4d17-af2c-8a94e896239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:14.546496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45cd76-1925-4d17-af2c-8a94e896239f admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:14.546888 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45cd76-1925-4d17-af2c-8a94e896239f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:14.547161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45cd76-1925-4d17-af2c-8a94e896239f admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:14.547887 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45cd76-1925-4d17-af2c-8a94e896239f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:14.549882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45cd76-1925-4d17-af2c-8a94e896239f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:14.586251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1246/2536] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:14 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:14.590589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019526ca-9670-4fb4-ae25-15e5efffb9ca admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:14.604555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019526ca-9670-4fb4-ae25-15e5efffb9ca admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:14.704240 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9865646-abb4-4799-b9e5-1772661e5adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:14.705009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9865646-abb4-4799-b9e5-1772661e5adb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:14.705301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9865646-abb4-4799-b9e5-1772661e5adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:14.705584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9865646-abb4-4799-b9e5-1772661e5adb admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:14.706374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9865646-abb4-4799-b9e5-1772661e5adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:14.708478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9865646-abb4-4799-b9e5-1772661e5adb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:14.745231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45cd76-1925-4d17-af2c-8a94e896239f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:14.745923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1291/2537] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:14 2022] POST /identity/v3/users => generated 281 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:14.763124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45cd76-1925-4d17-af2c-8a94e896239f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:14.775498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510e940-7b50-408c-ad15-02181555ce0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:14.776739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510e940-7b50-408c-ad15-02181555ce0e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:14.777361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510e940-7b50-408c-ad15-02181555ce0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:14.777953 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510e940-7b50-408c-ad15-02181555ce0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:14.779353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510e940-7b50-408c-ad15-02181555ce0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:14.781499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510e940-7b50-408c-ad15-02181555ce0e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:14.799193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1247/2538] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:14.804186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9865646-abb4-4799-b9e5-1772661e5adb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:14.825549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9865646-abb4-4799-b9e5-1772661e5adb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:14.834868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eff78b-4014-4228-939f-937e9219e5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:14.835626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eff78b-4014-4228-939f-937e9219e5ce admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:14.836034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eff78b-4014-4228-939f-937e9219e5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:14.836327 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eff78b-4014-4228-939f-937e9219e5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:14.837273 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eff78b-4014-4228-939f-937e9219e5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:14.839334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eff78b-4014-4228-939f-937e9219e5ce admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:14.858574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510e940-7b50-408c-ad15-02181555ce0e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:14.869429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1292/2539] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:14.875075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510e940-7b50-408c-ad15-02181555ce0e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:14.885119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a2490-9f09-4847-9f61-63cc28be1402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:14.886204 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a2490-9f09-4847-9f61-63cc28be1402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:14.886450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a2490-9f09-4847-9f61-63cc28be1402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:14.886933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a2490-9f09-4847-9f61-63cc28be1402 admin admin] PATH_INFO: `/v3/projects/f1c899c80cad425dae802f5787038041/users/cba5f44018c64f34bebfb2ec513a317a/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:14.908201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a2490-9f09-4847-9f61-63cc28be1402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c899c80cad425dae802f5787038041, user_id=cba5f44018c64f34bebfb2ec513a317a, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:14.911231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a2490-9f09-4847-9f61-63cc28be1402 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:14.941078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1248/2540] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:14 2022] PUT /identity/v3/projects/f1c899c80cad425dae802f5787038041/users/cba5f44018c64f34bebfb2ec513a317a/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:14.967537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e3d97-810f-4562-9d49-e715520b2e60 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:14.967832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e3d97-810f-4562-9d49-e715520b2e60 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:14.968765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e3d97-810f-4562-9d49-e715520b2e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:15.013466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-982e3d97-810f-4562-9d49-e715520b2e60 None None] MFA Rules not processed for user `cba5f44018c64f34bebfb2ec513a317a`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:15.015213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-982e3d97-810f-4562-9d49-e715520b2e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:15.112455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1293/2541] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:15.119412 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a2490-9f09-4847-9f61-63cc28be1402 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:15.196768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:15.263214 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a2490-9f09-4847-9f61-63cc28be1402 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:15.273138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362a4d8-b70c-4099-8929-54c01a164baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:15.274841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362a4d8-b70c-4099-8929-54c01a164baa admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:15.275239 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362a4d8-b70c-4099-8929-54c01a164baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:15.276521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362a4d8-b70c-4099-8929-54c01a164baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:15.276676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362a4d8-b70c-4099-8929-54c01a164baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:15.280983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362a4d8-b70c-4099-8929-54c01a164baa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:15.327012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1249/2542] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:15 2022] POST /identity/v3/projects => generated 363 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:15.327753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e3d97-810f-4562-9d49-e715520b2e60 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:15.346993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e3d97-810f-4562-9d49-e715520b2e60 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:15.444182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcc3bc-f084-4c64-925c-3b652833c72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:15.444182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcc3bc-f084-4c64-925c-3b652833c72d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:15.444182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcc3bc-f084-4c64-925c-3b652833c72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:15.444182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcc3bc-f084-4c64-925c-3b652833c72d admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:15.445148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcc3bc-f084-4c64-925c-3b652833c72d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:15.447840 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcc3bc-f084-4c64-925c-3b652833c72d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:15.473491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:15.488902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1294/2543] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:15 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:15.489841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362a4d8-b70c-4099-8929-54c01a164baa admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:15.503300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362a4d8-b70c-4099-8929-54c01a164baa admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:15.509072 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a4cc4-d64d-4e25-803a-380d901eecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:15.510300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a4cc4-d64d-4e25-803a-380d901eecb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:15.512874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a4cc4-d64d-4e25-803a-380d901eecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:15.513196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a4cc4-d64d-4e25-803a-380d901eecb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:15.514022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a4cc4-d64d-4e25-803a-380d901eecb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:15.515733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a4cc4-d64d-4e25-803a-380d901eecb1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:15.542754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1250/2544] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:15.547424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcc3bc-f084-4c64-925c-3b652833c72d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:15.575203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcc3bc-f084-4c64-925c-3b652833c72d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:15.581993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd3e51-3ef9-48b1-9c44-eb458ad5f5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:15.583189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd3e51-3ef9-48b1-9c44-eb458ad5f5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:15.583587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd3e51-3ef9-48b1-9c44-eb458ad5f5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:15.583986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd3e51-3ef9-48b1-9c44-eb458ad5f5f5 admin admin] PATH_INFO: `/v3/projects/f564b3c83fd84daa83dd52420048cce2/users/b4ee69d474b948b1bc4b6f679c061b2b/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:15.614698 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd3e51-3ef9-48b1-9c44-eb458ad5f5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f564b3c83fd84daa83dd52420048cce2, user_id=b4ee69d474b948b1bc4b6f679c061b2b, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:15.617525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd3e51-3ef9-48b1-9c44-eb458ad5f5f5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:15.652377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1295/2545] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:15 2022] PUT /identity/v3/projects/f564b3c83fd84daa83dd52420048cce2/users/b4ee69d474b948b1bc4b6f679c061b2b/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:15.654287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a4cc4-d64d-4e25-803a-380d901eecb1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:15.674706 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a4cc4-d64d-4e25-803a-380d901eecb1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:15.787988 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2380590-c200-4e91-a195-8bfde3925f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:15.788780 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2380590-c200-4e91-a195-8bfde3925f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:15.788990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2380590-c200-4e91-a195-8bfde3925f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:15.790752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2380590-c200-4e91-a195-8bfde3925f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:15.792059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2380590-c200-4e91-a195-8bfde3925f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:15.794272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2380590-c200-4e91-a195-8bfde3925f3c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:15.812985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1251/2546] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:15.817265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd3e51-3ef9-48b1-9c44-eb458ad5f5f5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:15.834566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd3e51-3ef9-48b1-9c44-eb458ad5f5f5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:15.845765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca341710-064e-487d-a1ae-443bd52aa948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:15.846677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca341710-064e-487d-a1ae-443bd52aa948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:15.846993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca341710-064e-487d-a1ae-443bd52aa948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:15.847279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca341710-064e-487d-a1ae-443bd52aa948 admin admin] PATH_INFO: `/v3/projects/f564b3c83fd84daa83dd52420048cce2/users/b4ee69d474b948b1bc4b6f679c061b2b/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:15.849804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca341710-064e-487d-a1ae-443bd52aa948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f564b3c83fd84daa83dd52420048cce2, user_id=b4ee69d474b948b1bc4b6f679c061b2b, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:15.853549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca341710-064e-487d-a1ae-443bd52aa948 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:15.871846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1296/2547] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:15 2022] PUT /identity/v3/projects/f564b3c83fd84daa83dd52420048cce2/users/b4ee69d474b948b1bc4b6f679c061b2b/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:15.891490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cce126-97a2-4590-9df9-38186d02e186 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:15.892238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cce126-97a2-4590-9df9-38186d02e186 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:15.892729 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cce126-97a2-4590-9df9-38186d02e186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:15.927826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f9cce126-97a2-4590-9df9-38186d02e186 None None] MFA Rules not processed for user `b4ee69d474b948b1bc4b6f679c061b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:15.929164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f9cce126-97a2-4590-9df9-38186d02e186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:16.052720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1252/2548] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:16.070361 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca341710-064e-487d-a1ae-443bd52aa948 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:16.073247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ca341710-064e-487d-a1ae-443bd52aa948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:16.191601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca341710-064e-487d-a1ae-443bd52aa948 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:16.196380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36c137-5eb7-4917-8269-a902baff06ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:16.197128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36c137-5eb7-4917-8269-a902baff06ea service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:16.197740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36c137-5eb7-4917-8269-a902baff06ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:16.198042 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36c137-5eb7-4917-8269-a902baff06ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:16.208654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36c137-5eb7-4917-8269-a902baff06ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:16.210232 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36c137-5eb7-4917-8269-a902baff06ea service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:16.221882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1297/2549] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:16 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:16.313021 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cce126-97a2-4590-9df9-38186d02e186 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:16.325674 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cce126-97a2-4590-9df9-38186d02e186 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:16.332227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a37275-1c83-4ce9-9c1c-43c97b58b6bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:16.333093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a37275-1c83-4ce9-9c1c-43c97b58b6bd service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:16.334315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a37275-1c83-4ce9-9c1c-43c97b58b6bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:16.335217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a37275-1c83-4ce9-9c1c-43c97b58b6bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:16.382507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:16.461427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a37275-1c83-4ce9-9c1c-43c97b58b6bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:16.465770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a37275-1c83-4ce9-9c1c-43c97b58b6bd service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:16.498321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1253/2550] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:17.348084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36c137-5eb7-4917-8269-a902baff06ea service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:17.349613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4b36c137-5eb7-4917-8269-a902baff06ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:17.463281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36c137-5eb7-4917-8269-a902baff06ea service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:17.467741 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:17.468653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:17.468937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:17.469217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] PATH_INFO: `/v3/users/3860c7cd45574d3a9c6eddfaa9697a7f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:17.470549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3860c7cd45574d3a9c6eddfaa9697a7f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:17.472526 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:17.532325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3860c7cd45574d3a9c6eddfaa9697a7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:17.542420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3860c7cd45574d3a9c6eddfaa9697a7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:17.543291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3860c7cd45574d3a9c6eddfaa9697a7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:17.554130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3860c7cd45574d3a9c6eddfaa9697a7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:17.557377 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:17.575634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3860c7cd45574d3a9c6eddfaa9697a7f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:17.585946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1298/2551] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:17 2022] DELETE /identity/v3/users/3860c7cd45574d3a9c6eddfaa9697a7f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:17.591589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17092cb-3b0f-4357-8314-f3dba427392b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:17.592298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17092cb-3b0f-4357-8314-f3dba427392b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:17.592298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17092cb-3b0f-4357-8314-f3dba427392b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:17.618207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f17092cb-3b0f-4357-8314-f3dba427392b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:17.619605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f17092cb-3b0f-4357-8314-f3dba427392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:17.716267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1254/2552] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:17.728625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:17.731120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:17.822884 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aba1fe-8e4d-4718-bf48-830dc568b5bf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:17.826810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5677bed-ea46-46bd-bf43-e07445aa322d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:17.827489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5677bed-ea46-46bd-bf43-e07445aa322d service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:17.827877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5677bed-ea46-46bd-bf43-e07445aa322d service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:17.828081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5677bed-ea46-46bd-bf43-e07445aa322d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:17.841082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5677bed-ea46-46bd-bf43-e07445aa322d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:17.844310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5677bed-ea46-46bd-bf43-e07445aa322d service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:17.865415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1299/2553] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:18.322927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17092cb-3b0f-4357-8314-f3dba427392b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:18.332394 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f17092cb-3b0f-4357-8314-f3dba427392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:18.351535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17092cb-3b0f-4357-8314-f3dba427392b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:18.358224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:18.359716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:18.360069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:18.360414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] PATH_INFO: `/v3/projects/fbea9422f6fe4521a49803cfe7e43e4b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:18.362753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbea9422f6fe4521a49803cfe7e43e4b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:18.365451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:18.455793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbea9422f6fe4521a49803cfe7e43e4b', 'request_id': 'req-87b9dc76-4b41-4c95-bb48-d00511ff68ae'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.469856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbea9422f6fe4521a49803cfe7e43e4b', 'request_id': 'req-87b9dc76-4b41-4c95-bb48-d00511ff68ae'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.484223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbea9422f6fe4521a49803cfe7e43e4b', 'request_id': 'req-87b9dc76-4b41-4c95-bb48-d00511ff68ae'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.502369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] The token cache is being invalidate because project fbea9422f6fe4521a49803cfe7e43e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:18.503640 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.504587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.506637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1255/2554] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:18 2022] DELETE /identity/v3/projects/fbea9422f6fe4521a49803cfe7e43e4b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:18.509911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5677bed-ea46-46bd-bf43-e07445aa322d service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:18.511751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:18.513408 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:18.514584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e5677bed-ea46-46bd-bf43-e07445aa322d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:18.577535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:18.612605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9dc76-4b41-4c95-bb48-d00511ff68ae admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:18.620435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:18.628539 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:18.629398 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:18.629592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] PATH_INFO: `/v3/users/e3dca51a476342688db661374ae5800c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:18.631409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3dca51a476342688db661374ae5800c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:18.634584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:18.696729 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5677bed-ea46-46bd-bf43-e07445aa322d service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:18.744604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3dca51a476342688db661374ae5800c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.767321 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3dca51a476342688db661374ae5800c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.769648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3dca51a476342688db661374ae5800c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.803740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3dca51a476342688db661374ae5800c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.804745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3dca51a476342688db661374ae5800c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:18.828334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db358004-ef72-42b4-9c05-cc9d3a1560ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:18.829068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db358004-ef72-42b4-9c05-cc9d3a1560ef service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:18.829457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db358004-ef72-42b4-9c05-cc9d3a1560ef service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:18.829797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db358004-ef72-42b4-9c05-cc9d3a1560ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:18.832083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1256/2555] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:18 2022] DELETE /identity/v3/users/e3dca51a476342688db661374ae5800c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:18.832403 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-db358004-ef72-42b4-9c05-cc9d3a1560ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:18.851236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db358004-ef72-42b4-9c05-cc9d3a1560ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:18.856220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db358004-ef72-42b4-9c05-cc9d3a1560ef service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:18.879182 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1300/2556] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:30:19.342818 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:19.344557 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:19.450812 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0748-cf08-45ef-8f0d-2cf74426a97e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:19.458209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:19.464442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:19.465053 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:19.465651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] PATH_INFO: `/v3/projects/e677549e4c1146e28d89ce4e0f9d6ec1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:19.475217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e677549e4c1146e28d89ce4e0f9d6ec1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:19.481280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:19.625956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e677549e4c1146e28d89ce4e0f9d6ec1', 'request_id': 'req-705d2141-3a4d-4c41-afcf-7d29db86def3'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:19.639812 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e677549e4c1146e28d89ce4e0f9d6ec1', 'request_id': 'req-705d2141-3a4d-4c41-afcf-7d29db86def3'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:19.652267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:19.657080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e677549e4c1146e28d89ce4e0f9d6ec1', 'request_id': 'req-705d2141-3a4d-4c41-afcf-7d29db86def3'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:19.677123 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] The token cache is being invalidate because project e677549e4c1146e28d89ce4e0f9d6ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:19.677599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:19.679546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:19.683926 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1257/2557] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:19 2022] DELETE /identity/v3/projects/e677549e4c1146e28d89ce4e0f9d6ec1 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:19.692397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca3416-32a4-4685-96b7-f3e8dcfd6d39 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:19.693121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca3416-32a4-4685-96b7-f3e8dcfd6d39 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:19.693121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca3416-32a4-4685-96b7-f3e8dcfd6d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:19.702388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:19.706893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:19.724993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e0ca3416-32a4-4685-96b7-f3e8dcfd6d39 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:19.726298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e0ca3416-32a4-4685-96b7-f3e8dcfd6d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:19.745656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:19.859118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d2141-3a4d-4c41-afcf-7d29db86def3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:19.868235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:19.871147 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:19.878743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:19.879287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] PATH_INFO: `/v3/users/cba5f44018c64f34bebfb2ec513a317a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:19.886367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cba5f44018c64f34bebfb2ec513a317a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:19.890577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:19.917518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1301/2558] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:19.932773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae9edc-0367-40ee-8e81-42cfc5ee1120 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:19.933208 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae9edc-0367-40ee-8e81-42cfc5ee1120 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:19.933407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae9edc-0367-40ee-8e81-42cfc5ee1120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:19.989560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b1ae9edc-0367-40ee-8e81-42cfc5ee1120 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:19.996378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b1ae9edc-0367-40ee-8e81-42cfc5ee1120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:20.016516 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1302/2559] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:20.017537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae9edc-0367-40ee-8e81-42cfc5ee1120 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.030510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae9edc-0367-40ee-8e81-42cfc5ee1120 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.036600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043c458-a481-466e-81e8-95e2604131f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.037335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c458-a481-466e-81e8-95e2604131f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.037618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c458-a481-466e-81e8-95e2604131f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.037943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c458-a481-466e-81e8-95e2604131f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.038907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043c458-a481-466e-81e8-95e2604131f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:20.039267 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043c458-a481-466e-81e8-95e2604131f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:20.042218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043c458-a481-466e-81e8-95e2604131f9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:20.048499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cba5f44018c64f34bebfb2ec513a317a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:20.061737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1303/2560] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:20.061948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cba5f44018c64f34bebfb2ec513a317a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:20.063086 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043c458-a481-466e-81e8-95e2604131f9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.063836 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cba5f44018c64f34bebfb2ec513a317a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:20.085877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043c458-a481-466e-81e8-95e2604131f9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.089913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cba5f44018c64f34bebfb2ec513a317a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:20.091666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d3140ec9-88b3-4961-a56a-cd568b5e0e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cba5f44018c64f34bebfb2ec513a317a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:20.096883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5b5aa-16ae-4229-aab1-be9a835b1812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.097950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5b5aa-16ae-4229-aab1-be9a835b1812 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.098317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5b5aa-16ae-4229-aab1-be9a835b1812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.098607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5b5aa-16ae-4229-aab1-be9a835b1812 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.099339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5b5aa-16ae-4229-aab1-be9a835b1812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:20.102753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5b5aa-16ae-4229-aab1-be9a835b1812 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:20.121882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1258/2561] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:19 2022] DELETE /identity/v3/users/cba5f44018c64f34bebfb2ec513a317a => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:20.131078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520273b7-33b1-462e-80ab-7afeccea1852 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.131922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520273b7-33b1-462e-80ab-7afeccea1852 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.131922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520273b7-33b1-462e-80ab-7afeccea1852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.158221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-520273b7-33b1-462e-80ab-7afeccea1852 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:20.160848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-520273b7-33b1-462e-80ab-7afeccea1852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:20.172614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1304/2562] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:20 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:20.175323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5b5aa-16ae-4229-aab1-be9a835b1812 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.178508 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-14f5b5aa-16ae-4229-aab1-be9a835b1812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:20.296304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1259/2563] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:20.296582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5b5aa-16ae-4229-aab1-be9a835b1812 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.307117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0137da4-531b-41ac-bd80-41c9c138092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.308453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0137da4-531b-41ac-bd80-41c9c138092b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.308891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0137da4-531b-41ac-bd80-41c9c138092b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.309304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0137da4-531b-41ac-bd80-41c9c138092b admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.309471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520273b7-33b1-462e-80ab-7afeccea1852 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.310147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0137da4-531b-41ac-bd80-41c9c138092b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:20.311083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-520273b7-33b1-462e-80ab-7afeccea1852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:20.312194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0137da4-531b-41ac-bd80-41c9c138092b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:20.327898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:20.338185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1305/2564] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:20 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:20.343406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0137da4-531b-41ac-bd80-41c9c138092b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.355979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0137da4-531b-41ac-bd80-41c9c138092b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.361204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6a461-db32-459b-8524-45417dd7c6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.361927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6a461-db32-459b-8524-45417dd7c6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.362241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6a461-db32-459b-8524-45417dd7c6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.362552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6a461-db32-459b-8524-45417dd7c6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.363308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6a461-db32-459b-8524-45417dd7c6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:20.364956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6a461-db32-459b-8524-45417dd7c6a1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:20.377093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1306/2565] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:20.381357 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6a461-db32-459b-8524-45417dd7c6a1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.392922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520273b7-33b1-462e-80ab-7afeccea1852 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.393210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6a461-db32-459b-8524-45417dd7c6a1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.396915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b8fa5-b84a-4a13-9480-14ff95fac71a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.397609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b8fa5-b84a-4a13-9480-14ff95fac71a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.397930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b8fa5-b84a-4a13-9480-14ff95fac71a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.398203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b8fa5-b84a-4a13-9480-14ff95fac71a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.400542 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363734fc-fa86-4ba9-aafb-63bff3474fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.401453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363734fc-fa86-4ba9-aafb-63bff3474fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.401878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363734fc-fa86-4ba9-aafb-63bff3474fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.402246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363734fc-fa86-4ba9-aafb-63bff3474fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.403019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363734fc-fa86-4ba9-aafb-63bff3474fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:20.405358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363734fc-fa86-4ba9-aafb-63bff3474fa5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:20.409830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b8fa5-b84a-4a13-9480-14ff95fac71a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:20.411510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b8fa5-b84a-4a13-9480-14ff95fac71a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:20.421882 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1307/2566] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:20.426807 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363734fc-fa86-4ba9-aafb-63bff3474fa5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.428042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1260/2567] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:30:20.445762 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363734fc-fa86-4ba9-aafb-63bff3474fa5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.454191 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfce893-0897-4a2f-b079-d0b37cae666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.455765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfce893-0897-4a2f-b079-d0b37cae666f admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.456265 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfce893-0897-4a2f-b079-d0b37cae666f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.456751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfce893-0897-4a2f-b079-d0b37cae666f admin admin] PATH_INFO: `/v3/projects/66b907ecb0c5483d979d8c7c5e93ed59/users/1f8404e8c99d48ac9c0bb4787eb80575/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.478297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfce893-0897-4a2f-b079-d0b37cae666f admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b907ecb0c5483d979d8c7c5e93ed59, user_id=1f8404e8c99d48ac9c0bb4787eb80575, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:20.483245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfce893-0897-4a2f-b079-d0b37cae666f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:20.503128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1308/2568] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:20 2022] PUT /identity/v3/projects/66b907ecb0c5483d979d8c7c5e93ed59/users/1f8404e8c99d48ac9c0bb4787eb80575/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:20.521137 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861414f5-05ea-4487-b503-fba13a23e55e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.521593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861414f5-05ea-4487-b503-fba13a23e55e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.522082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861414f5-05ea-4487-b503-fba13a23e55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.573686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-861414f5-05ea-4487-b503-fba13a23e55e None None] MFA Rules not processed for user `1f8404e8c99d48ac9c0bb4787eb80575`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:20.575854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-861414f5-05ea-4487-b503-fba13a23e55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:20.686561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1261/2569] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:20.692885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfce893-0897-4a2f-b079-d0b37cae666f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.725167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:20.786892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfce893-0897-4a2f-b079-d0b37cae666f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.792854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921a644-dfd4-44ec-9600-65a8ba87f067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.793579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921a644-dfd4-44ec-9600-65a8ba87f067 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.793968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921a644-dfd4-44ec-9600-65a8ba87f067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.794291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921a644-dfd4-44ec-9600-65a8ba87f067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.795083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921a644-dfd4-44ec-9600-65a8ba87f067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:20.797225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921a644-dfd4-44ec-9600-65a8ba87f067 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:20.832287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1309/2570] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:20 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:20.838285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861414f5-05ea-4487-b503-fba13a23e55e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.844658 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921a644-dfd4-44ec-9600-65a8ba87f067 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:20.851430 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b921a644-dfd4-44ec-9600-65a8ba87f067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:20.876440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861414f5-05ea-4487-b503-fba13a23e55e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.957237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921a644-dfd4-44ec-9600-65a8ba87f067 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:20.965039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.966052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.966337 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.966787 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] PATH_INFO: `/v3/projects/f1c899c80cad425dae802f5787038041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.968233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1c899c80cad425dae802f5787038041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:20.970419 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:20.997023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ff60-08c1-451e-ba32-4d58b91a5931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:20.997795 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ff60-08c1-451e-ba32-4d58b91a5931 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:20.998056 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ff60-08c1-451e-ba32-4d58b91a5931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:20.998461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ff60-08c1-451e-ba32-4d58b91a5931 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:20.999174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107ff60-08c1-451e-ba32-4d58b91a5931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:21.001301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107ff60-08c1-451e-ba32-4d58b91a5931 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:21.029751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1262/2571] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:20 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:21.035454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ff60-08c1-451e-ba32-4d58b91a5931 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:21.050763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ff60-08c1-451e-ba32-4d58b91a5931 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:21.062547 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c899c80cad425dae802f5787038041', 'request_id': 'req-a0cfe2a2-1b94-47d7-8b15-48be54afe421'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.073287 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c899c80cad425dae802f5787038041', 'request_id': 'req-a0cfe2a2-1b94-47d7-8b15-48be54afe421'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.083973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c899c80cad425dae802f5787038041', 'request_id': 'req-a0cfe2a2-1b94-47d7-8b15-48be54afe421'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.099977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:21.112283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] The token cache is being invalidate because project f1c899c80cad425dae802f5787038041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:21.112583 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.113270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.114918 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1310/2572] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:20 2022] DELETE /identity/v3/projects/f1c899c80cad425dae802f5787038041 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:21.119251 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:21.120883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:21.146331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abdd86-e88e-40fd-83c7-0689e5388132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:21.147013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abdd86-e88e-40fd-83c7-0689e5388132 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:21.147296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abdd86-e88e-40fd-83c7-0689e5388132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:21.147570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abdd86-e88e-40fd-83c7-0689e5388132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:21.148309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abdd86-e88e-40fd-83c7-0689e5388132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:21.150599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abdd86-e88e-40fd-83c7-0689e5388132 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:21.164259 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1263/2573] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:21.169098 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abdd86-e88e-40fd-83c7-0689e5388132 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:21.170974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-36abdd86-e88e-40fd-83c7-0689e5388132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:21.174855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:21.184634 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abdd86-e88e-40fd-83c7-0689e5388132 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:21.189531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79944ec4-86bb-4d97-bb9c-2d9d23c87e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:21.190244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79944ec4-86bb-4d97-bb9c-2d9d23c87e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:21.190528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79944ec4-86bb-4d97-bb9c-2d9d23c87e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:21.190804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79944ec4-86bb-4d97-bb9c-2d9d23c87e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:21.191536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79944ec4-86bb-4d97-bb9c-2d9d23c87e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:21.193149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79944ec4-86bb-4d97-bb9c-2d9d23c87e1c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:21.205639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfe2a2-1b94-47d7-8b15-48be54afe421 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:21.207263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1264/2574] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:21.211816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:21.212073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79944ec4-86bb-4d97-bb9c-2d9d23c87e1c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:21.212682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:21.213246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:21.213539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] PATH_INFO: `/v3/users/b4ee69d474b948b1bc4b6f679c061b2b` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:21.214903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ee69d474b948b1bc4b6f679c061b2b)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:21.216869 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:21.225469 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79944ec4-86bb-4d97-bb9c-2d9d23c87e1c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:21.235096 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a3fb9-44f0-4284-8e3f-e2cbfd13fa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:21.236251 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a3fb9-44f0-4284-8e3f-e2cbfd13fa2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:21.236642 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a3fb9-44f0-4284-8e3f-e2cbfd13fa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:21.237582 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a3fb9-44f0-4284-8e3f-e2cbfd13fa2d admin admin] PATH_INFO: `/v3/projects/8fdfac46a098465c9b3c4d6e68962fb8/users/8555c56ee5af48f5a81695625d2b7172/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:21.261779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a3fb9-44f0-4284-8e3f-e2cbfd13fa2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fdfac46a098465c9b3c4d6e68962fb8, user_id=8555c56ee5af48f5a81695625d2b7172, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:21.264458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a3fb9-44f0-4284-8e3f-e2cbfd13fa2d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:21.281190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1265/2575] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:21 2022] PUT /identity/v3/projects/8fdfac46a098465c9b3c4d6e68962fb8/users/8555c56ee5af48f5a81695625d2b7172/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:21.284169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ee69d474b948b1bc4b6f679c061b2b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.292933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c720266-24ec-4562-a20d-bf98212685ab None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:21.293249 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c720266-24ec-4562-a20d-bf98212685ab None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:21.293536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c720266-24ec-4562-a20d-bf98212685ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:21.296122 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ee69d474b948b1bc4b6f679c061b2b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.296898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ee69d474b948b1bc4b6f679c061b2b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.308728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ee69d474b948b1bc4b6f679c061b2b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.323722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ee69d474b948b1bc4b6f679c061b2b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:21.333766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1311/2576] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:21 2022] DELETE /identity/v3/users/b4ee69d474b948b1bc4b6f679c061b2b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:21.341602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2c720266-24ec-4562-a20d-bf98212685ab None None] MFA Rules not processed for user `8555c56ee5af48f5a81695625d2b7172`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:21.342916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2c720266-24ec-4562-a20d-bf98212685ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:21.463128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1266/2577] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:21.468197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:21.469930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:21.584672 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e300f51-5572-417d-bbfd-5ec9b5cd8257 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:21.593108 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba77e32-1557-4bee-9e7c-f8530804b777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:21.594260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba77e32-1557-4bee-9e7c-f8530804b777 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:21.594544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba77e32-1557-4bee-9e7c-f8530804b777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:21.594907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba77e32-1557-4bee-9e7c-f8530804b777 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:21.596020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba77e32-1557-4bee-9e7c-f8530804b777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:21.599077 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba77e32-1557-4bee-9e7c-f8530804b777 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:21.642614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1312/2578] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:21 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:21.647115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c720266-24ec-4562-a20d-bf98212685ab None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:21.663807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c720266-24ec-4562-a20d-bf98212685ab None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:21.761300 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba77e32-1557-4bee-9e7c-f8530804b777 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:21.765890 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9ba77e32-1557-4bee-9e7c-f8530804b777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:21.856492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:21.860486 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a7735-18fa-4996-afbc-e9eceae1324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:21.861741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a7735-18fa-4996-afbc-e9eceae1324f admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:21.862435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a7735-18fa-4996-afbc-e9eceae1324f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:21.863139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a7735-18fa-4996-afbc-e9eceae1324f admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:21.864440 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a7735-18fa-4996-afbc-e9eceae1324f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:21.868870 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a7735-18fa-4996-afbc-e9eceae1324f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:21.884935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba77e32-1557-4bee-9e7c-f8530804b777 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:21.890873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:21.891896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:21.892212 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:21.893034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] PATH_INFO: `/v3/projects/f564b3c83fd84daa83dd52420048cce2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:21.894025 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f564b3c83fd84daa83dd52420048cce2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:21.896934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:21.908952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1267/2579] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:21 2022] POST /identity/v3/users => generated 294 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:21.910768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a7735-18fa-4996-afbc-e9eceae1324f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:21.921585 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a7735-18fa-4996-afbc-e9eceae1324f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:21.926929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ebbca-cb31-4aa5-8929-9af62a7c5941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:21.927622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ebbca-cb31-4aa5-8929-9af62a7c5941 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:21.927908 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ebbca-cb31-4aa5-8929-9af62a7c5941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:21.928186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ebbca-cb31-4aa5-8929-9af62a7c5941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:21.928968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ebbca-cb31-4aa5-8929-9af62a7c5941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:21.931202 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ebbca-cb31-4aa5-8929-9af62a7c5941 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:21.945922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1268/2580] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:21.955872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ebbca-cb31-4aa5-8929-9af62a7c5941 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:21.968499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:21.978762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ebbca-cb31-4aa5-8929-9af62a7c5941 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:21.986751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f564b3c83fd84daa83dd52420048cce2', 'request_id': 'req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:22.006318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f564b3c83fd84daa83dd52420048cce2', 'request_id': 'req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:22.028207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f564b3c83fd84daa83dd52420048cce2', 'request_id': 'req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:22.045867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] The token cache is being invalidate because project f564b3c83fd84daa83dd52420048cce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:22.046549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:22.048549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4d1dd2fa-f1c7-4d0f-9c5a-fdbeee4f2c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:22.050975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1313/2581] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:21 2022] DELETE /identity/v3/projects/f564b3c83fd84daa83dd52420048cce2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:22.057867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615298b6-9b2b-4db3-99f5-47b8fd294d9b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.058179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615298b6-9b2b-4db3-99f5-47b8fd294d9b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.058462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615298b6-9b2b-4db3-99f5-47b8fd294d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.085946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-615298b6-9b2b-4db3-99f5-47b8fd294d9b None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:22.087774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-615298b6-9b2b-4db3-99f5-47b8fd294d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:22.097381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe8458-88a5-48ae-8c8d-78f36dccb734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:22.098483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe8458-88a5-48ae-8c8d-78f36dccb734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.098642 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe8458-88a5-48ae-8c8d-78f36dccb734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.098883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe8458-88a5-48ae-8c8d-78f36dccb734 admin admin] PATH_INFO: `/v3/projects/b42aedc3113e493e9d4ab266aec24e3a/users/63b6c81f274542f2b4843bf16e1e0f88/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.122348 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe8458-88a5-48ae-8c8d-78f36dccb734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42aedc3113e493e9d4ab266aec24e3a, user_id=63b6c81f274542f2b4843bf16e1e0f88, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:22.126600 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe8458-88a5-48ae-8c8d-78f36dccb734 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:22.154468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1269/2582] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:21 2022] PUT /identity/v3/projects/b42aedc3113e493e9d4ab266aec24e3a/users/63b6c81f274542f2b4843bf16e1e0f88/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:22.158387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe8458-88a5-48ae-8c8d-78f36dccb734 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:22.162975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-52fe8458-88a5-48ae-8c8d-78f36dccb734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:22.253412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1314/2583] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:22.266033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c99ccf-d788-44c8-b7d5-549578911250 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.266316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c99ccf-d788-44c8-b7d5-549578911250 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.266684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c99ccf-d788-44c8-b7d5-549578911250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.299887 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-40c99ccf-d788-44c8-b7d5-549578911250 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:22.301145 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-40c99ccf-d788-44c8-b7d5-549578911250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:22.305841 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe8458-88a5-48ae-8c8d-78f36dccb734 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:22.314582 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50de934-cad6-45c4-95b6-a5e9779fd94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:22.315831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50de934-cad6-45c4-95b6-a5e9779fd94a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.316121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1315/2584] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:22.316776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50de934-cad6-45c4-95b6-a5e9779fd94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.317438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50de934-cad6-45c4-95b6-a5e9779fd94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.318402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137c24-0218-48a1-b5a5-7988a54aa0a8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.318830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50de934-cad6-45c4-95b6-a5e9779fd94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:22.318966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137c24-0218-48a1-b5a5-7988a54aa0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.319290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137c24-0218-48a1-b5a5-7988a54aa0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.322553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50de934-cad6-45c4-95b6-a5e9779fd94a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:22.343839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1270/2585] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:22.346381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50de934-cad6-45c4-95b6-a5e9779fd94a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:22.356638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7e137c24-0218-48a1-b5a5-7988a54aa0a8 None None] MFA Rules not processed for user `bb0da8665c1b44f79611129ee1337bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:22.358366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7e137c24-0218-48a1-b5a5-7988a54aa0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:22.364243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50de934-cad6-45c4-95b6-a5e9779fd94a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:22.372320 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ba6ae-eb82-4218-99e8-d0566e272435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:22.373288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ba6ae-eb82-4218-99e8-d0566e272435 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.373784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ba6ae-eb82-4218-99e8-d0566e272435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.374277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ba6ae-eb82-4218-99e8-d0566e272435 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.375576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ba6ae-eb82-4218-99e8-d0566e272435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:22.376077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ba6ae-eb82-4218-99e8-d0566e272435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:22.378985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ba6ae-eb82-4218-99e8-d0566e272435 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:22.407174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1271/2586] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:22.408165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ba6ae-eb82-4218-99e8-d0566e272435 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:22.420456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ba6ae-eb82-4218-99e8-d0566e272435 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:22.425812 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9475ac-8108-40c1-b87f-20b2bc735231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:22.427307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9475ac-8108-40c1-b87f-20b2bc735231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.427805 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9475ac-8108-40c1-b87f-20b2bc735231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.428281 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9475ac-8108-40c1-b87f-20b2bc735231 admin admin] PATH_INFO: `/v3/projects/b42aedc3113e493e9d4ab266aec24e3a/users/63b6c81f274542f2b4843bf16e1e0f88/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.432236 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9475ac-8108-40c1-b87f-20b2bc735231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42aedc3113e493e9d4ab266aec24e3a, user_id=63b6c81f274542f2b4843bf16e1e0f88, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:22.435198 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9475ac-8108-40c1-b87f-20b2bc735231 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:22.451119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1272/2587] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:22 2022] PUT /identity/v3/projects/b42aedc3113e493e9d4ab266aec24e3a/users/63b6c81f274542f2b4843bf16e1e0f88/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:22.452647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9475ac-8108-40c1-b87f-20b2bc735231 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:22.473584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1316/2588] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 6 14:30:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:22.475563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1d2c1-6293-43c2-94fd-ab299d2bcf01 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.476056 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1d2c1-6293-43c2-94fd-ab299d2bcf01 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.476473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1d2c1-6293-43c2-94fd-ab299d2bcf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.514566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:22.530178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1bb1d2c1-6293-43c2-94fd-ab299d2bcf01 None None] MFA Rules not processed for user `63b6c81f274542f2b4843bf16e1e0f88`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:22.533090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1bb1d2c1-6293-43c2-94fd-ab299d2bcf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:22.556065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:22.567565 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9475ac-8108-40c1-b87f-20b2bc735231 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:22.577199 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea36e1f-2cb7-480c-830b-a0716fa0a1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:22.578430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea36e1f-2cb7-480c-830b-a0716fa0a1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.578927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea36e1f-2cb7-480c-830b-a0716fa0a1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.579883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea36e1f-2cb7-480c-830b-a0716fa0a1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.583599 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea36e1f-2cb7-480c-830b-a0716fa0a1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:22.585755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea36e1f-2cb7-480c-830b-a0716fa0a1d2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:22.628735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1273/2589] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:22 2022] POST /identity/v3/projects => generated 369 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:22.628735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea36e1f-2cb7-480c-830b-a0716fa0a1d2 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:22.628735 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-eea36e1f-2cb7-480c-830b-a0716fa0a1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:22.679047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1317/2590] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:22.685999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1d2c1-6293-43c2-94fd-ab299d2bcf01 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:22.704790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1d2c1-6293-43c2-94fd-ab299d2bcf01 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:22.774369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea36e1f-2cb7-480c-830b-a0716fa0a1d2 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:22.786924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087eafa-fe2e-455f-ae2b-3608926da8e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3571f3d3a0de4e9abe55ed84426143a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:22.788502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087eafa-fe2e-455f-ae2b-3608926da8e7 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.789328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087eafa-fe2e-455f-ae2b-3608926da8e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.790990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087eafa-fe2e-455f-ae2b-3608926da8e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.807709 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8979d28-7731-4d65-8c36-c3081b454d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:22.809978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8979d28-7731-4d65-8c36-c3081b454d0d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.810262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8979d28-7731-4d65-8c36-c3081b454d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.810555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8979d28-7731-4d65-8c36-c3081b454d0d admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.811294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8979d28-7731-4d65-8c36-c3081b454d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:22.813486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8979d28-7731-4d65-8c36-c3081b454d0d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:22.850415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1318/2591] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:22 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:22.851055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8979d28-7731-4d65-8c36-c3081b454d0d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:22.853227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b8979d28-7731-4d65-8c36-c3081b454d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:22.919287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087eafa-fe2e-455f-ae2b-3608926da8e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:22.920976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087eafa-fe2e-455f-ae2b-3608926da8e7 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:22.937189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1274/2592] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 6 14:30:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 14:30:22.939311 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087eafa-fe2e-455f-ae2b-3608926da8e7 service swift] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:22.962166 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087eafa-fe2e-455f-ae2b-3608926da8e7 service swift] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:22.971481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8979d28-7731-4d65-8c36-c3081b454d0d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:22.975467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f740df5-efcc-45d2-a363-0e52902c7c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:22.975686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eadbe7-53bd-4811-97ff-fce134141875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:22.976408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eadbe7-53bd-4811-97ff-fce134141875 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.976828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eadbe7-53bd-4811-97ff-fce134141875 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.977113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eadbe7-53bd-4811-97ff-fce134141875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.977575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f740df5-efcc-45d2-a363-0e52902c7c41 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:22.978087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f740df5-efcc-45d2-a363-0e52902c7c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:22.978456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f740df5-efcc-45d2-a363-0e52902c7c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:22.979373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f740df5-efcc-45d2-a363-0e52902c7c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:22.981403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f740df5-efcc-45d2-a363-0e52902c7c41 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:23.009726 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1275/2593] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:23.011209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f740df5-efcc-45d2-a363-0e52902c7c41 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:23.046683 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f740df5-efcc-45d2-a363-0e52902c7c41 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:23.062242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c9842-9ebf-4e94-ba00-1b73c4c97155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:23.062779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c9842-9ebf-4e94-ba00-1b73c4c97155 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:23.063071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c9842-9ebf-4e94-ba00-1b73c4c97155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:23.064112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c9842-9ebf-4e94-ba00-1b73c4c97155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:23.068315 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c9842-9ebf-4e94-ba00-1b73c4c97155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:23.074854 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c9842-9ebf-4e94-ba00-1b73c4c97155 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:23.084850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:23.094466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1276/2594] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:23.100879 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c9842-9ebf-4e94-ba00-1b73c4c97155 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:23.124458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c9842-9ebf-4e94-ba00-1b73c4c97155 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:23.135014 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707cf91-c97e-444a-a1c8-8867b3c9f131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:23.136173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707cf91-c97e-444a-a1c8-8867b3c9f131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:23.136330 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707cf91-c97e-444a-a1c8-8867b3c9f131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:23.136700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707cf91-c97e-444a-a1c8-8867b3c9f131 admin admin] PATH_INFO: `/v3/projects/e8d0902bce53491abeae88c797f6ebea/users/88aaeb481dbd4deeb5cb2a3f0edb5e70/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:23.153092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eadbe7-53bd-4811-97ff-fce134141875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:23.161305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eadbe7-53bd-4811-97ff-fce134141875 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:23.170336 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707cf91-c97e-444a-a1c8-8867b3c9f131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d0902bce53491abeae88c797f6ebea, user_id=88aaeb481dbd4deeb5cb2a3f0edb5e70, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:23.174643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707cf91-c97e-444a-a1c8-8867b3c9f131 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:23.177601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1319/2595] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:23.220588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1277/2596] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:23 2022] PUT /identity/v3/projects/e8d0902bce53491abeae88c797f6ebea/users/88aaeb481dbd4deeb5cb2a3f0edb5e70/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:23.238244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b925b-6de2-4b67-ba0b-fe587ee68a0e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:23.238586 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b925b-6de2-4b67-ba0b-fe587ee68a0e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:23.238903 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b925b-6de2-4b67-ba0b-fe587ee68a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:23.256385 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707cf91-c97e-444a-a1c8-8867b3c9f131 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:23.259496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a707cf91-c97e-444a-a1c8-8867b3c9f131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:23.307259 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3d1b925b-6de2-4b67-ba0b-fe587ee68a0e None None] MFA Rules not processed for user `88aaeb481dbd4deeb5cb2a3f0edb5e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:23.308534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3d1b925b-6de2-4b67-ba0b-fe587ee68a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:23.475188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1320/2597] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:23.483362 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b925b-6de2-4b67-ba0b-fe587ee68a0e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:23.579813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:23.607865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707cf91-c97e-444a-a1c8-8867b3c9f131 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:23.612680 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a3ed0-c4ba-40c6-8252-cf354a477616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:23.613207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a3ed0-c4ba-40c6-8252-cf354a477616 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:23.613497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a3ed0-c4ba-40c6-8252-cf354a477616 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:23.613992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a3ed0-c4ba-40c6-8252-cf354a477616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:23.616106 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-438a3ed0-c4ba-40c6-8252-cf354a477616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:23.721581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b925b-6de2-4b67-ba0b-fe587ee68a0e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:23.746678 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c83036-111d-49e8-a604-4b49692113d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:23.747342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c83036-111d-49e8-a604-4b49692113d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:23.747626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c83036-111d-49e8-a604-4b49692113d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:23.748049 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c83036-111d-49e8-a604-4b49692113d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:23.748802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c83036-111d-49e8-a604-4b49692113d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:23.751132 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c83036-111d-49e8-a604-4b49692113d6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:23.798956 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a3ed0-c4ba-40c6-8252-cf354a477616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:23.803488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a3ed0-c4ba-40c6-8252-cf354a477616 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:23.827597 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1278/2598] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:23.829538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1321/2599] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:23 2022] POST /identity/v3/projects => generated 365 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:23.835904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a3ed0-c4ba-40c6-8252-cf354a477616 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:23.855595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a3ed0-c4ba-40c6-8252-cf354a477616 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:23.952993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd0564-5124-406b-b774-b5d572d5aefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:23.954092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd0564-5124-406b-b774-b5d572d5aefc admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:23.954530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd0564-5124-406b-b774-b5d572d5aefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:23.954868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd0564-5124-406b-b774-b5d572d5aefc admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:23.955664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd0564-5124-406b-b774-b5d572d5aefc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:23.957882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd0564-5124-406b-b774-b5d572d5aefc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:24.016716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1279/2600] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:23 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:24.020916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c83036-111d-49e8-a604-4b49692113d6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:24.039421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c83036-111d-49e8-a604-4b49692113d6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:24.058176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31f5c3-6086-41f1-a7b6-c56630c3ae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:24.058975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31f5c3-6086-41f1-a7b6-c56630c3ae55 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:24.059347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31f5c3-6086-41f1-a7b6-c56630c3ae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:24.059626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31f5c3-6086-41f1-a7b6-c56630c3ae55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:24.060410 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31f5c3-6086-41f1-a7b6-c56630c3ae55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:24.062130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31f5c3-6086-41f1-a7b6-c56630c3ae55 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:24.093091 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1322/2601] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:24.094540 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd0564-5124-406b-b774-b5d572d5aefc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:24.121384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd0564-5124-406b-b774-b5d572d5aefc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:24.121384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7247-3cf9-47a8-a281-03ef0d8f6f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:24.121384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7247-3cf9-47a8-a281-03ef0d8f6f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:24.121384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7247-3cf9-47a8-a281-03ef0d8f6f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:24.121891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7247-3cf9-47a8-a281-03ef0d8f6f37 admin admin] PATH_INFO: `/v3/projects/29e023c42cc74e4d89a6e0b907a3dcfc/users/2017dbac7e484acfae55bb50197adcbf/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:24.155386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d7247-3cf9-47a8-a281-03ef0d8f6f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e023c42cc74e4d89a6e0b907a3dcfc, user_id=2017dbac7e484acfae55bb50197adcbf, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:24.163279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d7247-3cf9-47a8-a281-03ef0d8f6f37 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:24.207240 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1280/2602] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:24 2022] PUT /identity/v3/projects/29e023c42cc74e4d89a6e0b907a3dcfc/users/2017dbac7e484acfae55bb50197adcbf/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:24.227442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31f5c3-6086-41f1-a7b6-c56630c3ae55 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:24.261966 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31f5c3-6086-41f1-a7b6-c56630c3ae55 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:24.353395 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:24.419194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe5af3-5fc0-44fa-bb61-2c77ab9fddbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:24.420484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe5af3-5fc0-44fa-bb61-2c77ab9fddbb admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:24.420484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe5af3-5fc0-44fa-bb61-2c77ab9fddbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:24.420484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe5af3-5fc0-44fa-bb61-2c77ab9fddbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:24.421174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe5af3-5fc0-44fa-bb61-2c77ab9fddbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:24.422913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe5af3-5fc0-44fa-bb61-2c77ab9fddbb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:24.445897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1323/2603] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:24.450247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7247-3cf9-47a8-a281-03ef0d8f6f37 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:24.482075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7247-3cf9-47a8-a281-03ef0d8f6f37 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:24.495401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aee12-6f2b-4e28-ab35-815b81859358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:24.501013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aee12-6f2b-4e28-ab35-815b81859358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:24.519226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aee12-6f2b-4e28-ab35-815b81859358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:24.520427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aee12-6f2b-4e28-ab35-815b81859358 admin admin] PATH_INFO: `/v3/projects/29e023c42cc74e4d89a6e0b907a3dcfc/users/2017dbac7e484acfae55bb50197adcbf/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:24.528506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4aee12-6f2b-4e28-ab35-815b81859358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e023c42cc74e4d89a6e0b907a3dcfc, user_id=2017dbac7e484acfae55bb50197adcbf, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:24.537517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4aee12-6f2b-4e28-ab35-815b81859358 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:24.614175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1281/2604] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:24 2022] PUT /identity/v3/projects/29e023c42cc74e4d89a6e0b907a3dcfc/users/2017dbac7e484acfae55bb50197adcbf/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:24.636094 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb81b4-dc56-41ab-8305-e02216665923 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:24.637074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb81b4-dc56-41ab-8305-e02216665923 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:24.637925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb81b4-dc56-41ab-8305-e02216665923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:24.714568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-59cb81b4-dc56-41ab-8305-e02216665923 None None] MFA Rules not processed for user `2017dbac7e484acfae55bb50197adcbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:24.716589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-59cb81b4-dc56-41ab-8305-e02216665923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:24.858214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1324/2605] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:24.872991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aee12-6f2b-4e28-ab35-815b81859358 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:24.938317 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:24.952483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aee12-6f2b-4e28-ab35-815b81859358 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:24.957782 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be467239-7700-425e-bbbe-fcf96ef34779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:24.958748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be467239-7700-425e-bbbe-fcf96ef34779 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:24.958748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be467239-7700-425e-bbbe-fcf96ef34779 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:24.959093 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be467239-7700-425e-bbbe-fcf96ef34779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:24.971234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be467239-7700-425e-bbbe-fcf96ef34779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:24.972740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be467239-7700-425e-bbbe-fcf96ef34779 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:24.985082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1282/2606] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:30:28.201054 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb81b4-dc56-41ab-8305-e02216665923 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:28.202669 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be467239-7700-425e-bbbe-fcf96ef34779 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:28.448666 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be467239-7700-425e-bbbe-fcf96ef34779 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:28.471604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:28.478530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:28.479535 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:28.484414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] PATH_INFO: `/v3/users/88aaeb481dbd4deeb5cb2a3f0edb5e70` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:28.490406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] RBAC: Authorizing `identity:delete_user(user_id=88aaeb481dbd4deeb5cb2a3f0edb5e70)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:28.496861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:28.500491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb81b4-dc56-41ab-8305-e02216665923 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:28.516863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:28.517990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:28.518283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:28.518906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] PATH_INFO: `/v3/users/1f8404e8c99d48ac9c0bb4787eb80575` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:28.522079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f8404e8c99d48ac9c0bb4787eb80575)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:28.529112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:28.698814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f8404e8c99d48ac9c0bb4787eb80575'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.710222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88aaeb481dbd4deeb5cb2a3f0edb5e70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.720260 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f8404e8c99d48ac9c0bb4787eb80575'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.722682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f8404e8c99d48ac9c0bb4787eb80575'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.731811 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:28.741828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88aaeb481dbd4deeb5cb2a3f0edb5e70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.746178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88aaeb481dbd4deeb5cb2a3f0edb5e70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.746876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f8404e8c99d48ac9c0bb4787eb80575'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.801110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fd47d567-6a70-4e58-b2fa-372be0a3e5ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f8404e8c99d48ac9c0bb4787eb80575'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.806978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88aaeb481dbd4deeb5cb2a3f0edb5e70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.808750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c3bffec-1fce-4f21-9643-94ac7a34a99c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88aaeb481dbd4deeb5cb2a3f0edb5e70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:28.835220 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1325/2607] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:28 2022] DELETE /identity/v3/users/1f8404e8c99d48ac9c0bb4787eb80575 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:28.852439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf3755-2c44-4f6d-a24e-541e4c295ffd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:28.857356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf3755-2c44-4f6d-a24e-541e4c295ffd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:28.858564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf3755-2c44-4f6d-a24e-541e4c295ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:28.875344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1283/2608] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:28 2022] DELETE /identity/v3/users/88aaeb481dbd4deeb5cb2a3f0edb5e70 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:28.884809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da68359-11da-41bd-8f43-7f79eb3a8e87 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:28.886738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da68359-11da-41bd-8f43-7f79eb3a8e87 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:28.888244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da68359-11da-41bd-8f43-7f79eb3a8e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:28.928062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-76bf3755-2c44-4f6d-a24e-541e4c295ffd None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:28.933245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-76bf3755-2c44-4f6d-a24e-541e4c295ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:28.963534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5da68359-11da-41bd-8f43-7f79eb3a8e87 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:28.965270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5da68359-11da-41bd-8f43-7f79eb3a8e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:29.095566 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1284/2609] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:29.111911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da68359-11da-41bd-8f43-7f79eb3a8e87 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:29.114829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5da68359-11da-41bd-8f43-7f79eb3a8e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:29.127893 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:29.154726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1326/2610] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:29.175041 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf3755-2c44-4f6d-a24e-541e4c295ffd None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:29.178628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-76bf3755-2c44-4f6d-a24e-541e4c295ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:29.213521 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da68359-11da-41bd-8f43-7f79eb3a8e87 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:29.220931 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70ac20-44f6-446a-9fb4-85a8b0157bd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:29.221858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70ac20-44f6-446a-9fb4-85a8b0157bd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:29.224075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70ac20-44f6-446a-9fb4-85a8b0157bd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:29.224075 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70ac20-44f6-446a-9fb4-85a8b0157bd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:29.244733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70ac20-44f6-446a-9fb4-85a8b0157bd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:29.248918 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70ac20-44f6-446a-9fb4-85a8b0157bd6 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:29.270043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1285/2611] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:29.324597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf3755-2c44-4f6d-a24e-541e4c295ffd None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:29.333421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbc9ad-e0dd-440f-858f-f51cad12698f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:29.335759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbc9ad-e0dd-440f-858f-f51cad12698f service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:29.337634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbc9ad-e0dd-440f-858f-f51cad12698f service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:29.338725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbc9ad-e0dd-440f-858f-f51cad12698f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:29.374561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbc9ad-e0dd-440f-858f-f51cad12698f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:29.380002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbc9ad-e0dd-440f-858f-f51cad12698f service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:29.409795 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1327/2612] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:30.074223 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70ac20-44f6-446a-9fb4-85a8b0157bd6 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:30.078930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7b70ac20-44f6-446a-9fb4-85a8b0157bd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:30.119024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70ac20-44f6-446a-9fb4-85a8b0157bd6 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:30.138767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:30.142270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:30.143771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:30.143941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:30.165667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:30.247346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:30.253313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:30.312515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1286/2613] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:30.321347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbc9ad-e0dd-440f-858f-f51cad12698f service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:30.323302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b8dbc9ad-e0dd-440f-858f-f51cad12698f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:30.346270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbc9ad-e0dd-440f-858f-f51cad12698f service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:30.372636 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:30.382654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:30.383537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:30.383752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] PATH_INFO: `/v3/projects/e8d0902bce53491abeae88c797f6ebea` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:30.394468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d0902bce53491abeae88c797f6ebea)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:30.420060 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:30.535119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:30.536676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:30.564490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60521bc2-701f-43e2-8c98-530e44012abb service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:30.609089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d0902bce53491abeae88c797f6ebea', 'request_id': 'req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:30.630038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d0902bce53491abeae88c797f6ebea', 'request_id': 'req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:30.647544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d0902bce53491abeae88c797f6ebea', 'request_id': 'req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:30.697569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] The token cache is being invalidate because project e8d0902bce53491abeae88c797f6ebea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:30.702502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:30.703923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:30.705875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1328/2614] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:30 2022] DELETE /identity/v3/projects/e8d0902bce53491abeae88c797f6ebea => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:30.712817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:30.714767 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:30.722960 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:30.847997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd350ce7-c0ef-4b82-8411-3bc53821a3e6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:30.858067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:30.859433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:30.860234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:30.860761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] PATH_INFO: `/v3/users/2017dbac7e484acfae55bb50197adcbf` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:30.863477 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2017dbac7e484acfae55bb50197adcbf)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:30.872770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:30.893755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:30.896242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:30.897048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:30.897768 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] PATH_INFO: `/v3/projects/66b907ecb0c5483d979d8c7c5e93ed59` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:30.906234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b907ecb0c5483d979d8c7c5e93ed59)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:30.910170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:31.080907 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2017dbac7e484acfae55bb50197adcbf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.107184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2017dbac7e484acfae55bb50197adcbf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.108288 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2017dbac7e484acfae55bb50197adcbf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.141846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2017dbac7e484acfae55bb50197adcbf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.155647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b907ecb0c5483d979d8c7c5e93ed59', 'request_id': 'req-a860bbec-d5b6-4e6d-8fc1-7874585b944c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.178708 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b907ecb0c5483d979d8c7c5e93ed59', 'request_id': 'req-a860bbec-d5b6-4e6d-8fc1-7874585b944c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.203019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2017dbac7e484acfae55bb50197adcbf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.203988 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b907ecb0c5483d979d8c7c5e93ed59', 'request_id': 'req-a860bbec-d5b6-4e6d-8fc1-7874585b944c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.244994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1329/2615] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:30 2022] DELETE /identity/v3/users/2017dbac7e484acfae55bb50197adcbf => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:31.295629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] The token cache is being invalidate because project 66b907ecb0c5483d979d8c7c5e93ed59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:31.296748 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.301753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.308310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1287/2616] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:30 2022] DELETE /identity/v3/projects/66b907ecb0c5483d979d8c7c5e93ed59 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:31.313656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:31.316675 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:31.552528 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ce216-5c90-4d0e-895b-21957643fcf9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:31.574298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:31.575955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:31.576612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:31.576896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] PATH_INFO: `/v3/users/8555c56ee5af48f5a81695625d2b7172` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:31.579851 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8555c56ee5af48f5a81695625d2b7172)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:31.584737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:31.699249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:31.812949 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8555c56ee5af48f5a81695625d2b7172'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.858034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8555c56ee5af48f5a81695625d2b7172'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.859573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8555c56ee5af48f5a81695625d2b7172'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.880538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8555c56ee5af48f5a81695625d2b7172'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.938175 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9acf23ec-db94-4de5-bd86-09789ec5c447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8555c56ee5af48f5a81695625d2b7172'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:31.959384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1330/2617] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:31 2022] DELETE /identity/v3/users/8555c56ee5af48f5a81695625d2b7172 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:32.713520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:32.720077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:32.794167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:32.888936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860bbec-d5b6-4e6d-8fc1-7874585b944c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:33.024892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:33.027455 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:33.028272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:33.032151 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] PATH_INFO: `/v3/projects/29e023c42cc74e4d89a6e0b907a3dcfc` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:33.034711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29e023c42cc74e4d89a6e0b907a3dcfc)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:33.040938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:33.168184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29e023c42cc74e4d89a6e0b907a3dcfc', 'request_id': 'req-e87ebebc-b264-4477-8af7-7222c07623f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.183586 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29e023c42cc74e4d89a6e0b907a3dcfc', 'request_id': 'req-e87ebebc-b264-4477-8af7-7222c07623f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.196895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29e023c42cc74e4d89a6e0b907a3dcfc', 'request_id': 'req-e87ebebc-b264-4477-8af7-7222c07623f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.220241 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] The token cache is being invalidate because project 29e023c42cc74e4d89a6e0b907a3dcfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:33.220751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.226277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e87ebebc-b264-4477-8af7-7222c07623f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.235624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1288/2618] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:32 2022] DELETE /identity/v3/projects/29e023c42cc74e4d89a6e0b907a3dcfc => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:33.243369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672d552-f4c3-4d5b-991f-3e94ba4e3d49 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:33.244100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672d552-f4c3-4d5b-991f-3e94ba4e3d49 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:33.244100 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672d552-f4c3-4d5b-991f-3e94ba4e3d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:33.283249 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a672d552-f4c3-4d5b-991f-3e94ba4e3d49 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:33.285400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a672d552-f4c3-4d5b-991f-3e94ba4e3d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:33.429498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1331/2619] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:33.444095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533451-088d-44a0-ae78-9c295d93e132 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:33.453076 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533451-088d-44a0-ae78-9c295d93e132 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:33.453543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533451-088d-44a0-ae78-9c295d93e132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:33.511453 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-05533451-088d-44a0-ae78-9c295d93e132 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:33.513029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-05533451-088d-44a0-ae78-9c295d93e132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:33.558211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1289/2620] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:33.569538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672d552-f4c3-4d5b-991f-3e94ba4e3d49 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:33.603905 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672d552-f4c3-4d5b-991f-3e94ba4e3d49 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:33.623974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397ffa1-2ef2-45c2-acfc-004ff5db8b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:33.625109 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397ffa1-2ef2-45c2-acfc-004ff5db8b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:33.628915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397ffa1-2ef2-45c2-acfc-004ff5db8b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:33.629397 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397ffa1-2ef2-45c2-acfc-004ff5db8b7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:33.630299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533451-088d-44a0-ae78-9c295d93e132 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:33.630515 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397ffa1-2ef2-45c2-acfc-004ff5db8b7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:33.630780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397ffa1-2ef2-45c2-acfc-004ff5db8b7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:33.632187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-05533451-088d-44a0-ae78-9c295d93e132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:33.634203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397ffa1-2ef2-45c2-acfc-004ff5db8b7b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:33.660331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1332/2621] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:33.660950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533451-088d-44a0-ae78-9c295d93e132 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:33.670653 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397ffa1-2ef2-45c2-acfc-004ff5db8b7b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:33.670862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:33.672881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:33.673604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:33.674427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] PATH_INFO: `/v3/projects/8fdfac46a098465c9b3c4d6e68962fb8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:33.679848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fdfac46a098465c9b3c4d6e68962fb8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:33.682813 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:33.707318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397ffa1-2ef2-45c2-acfc-004ff5db8b7b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:33.729484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7f943-b0dd-4f0d-a467-fb1bc9113273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:33.732836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7f943-b0dd-4f0d-a467-fb1bc9113273 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:33.734342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7f943-b0dd-4f0d-a467-fb1bc9113273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:33.735090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7f943-b0dd-4f0d-a467-fb1bc9113273 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:33.737873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7f943-b0dd-4f0d-a467-fb1bc9113273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:33.742904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7f943-b0dd-4f0d-a467-fb1bc9113273 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:33.826118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fdfac46a098465c9b3c4d6e68962fb8', 'request_id': 'req-64a12d36-3e22-410c-950a-aa5935773752'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.830515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:33.843552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fdfac46a098465c9b3c4d6e68962fb8', 'request_id': 'req-64a12d36-3e22-410c-950a-aa5935773752'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.856387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fdfac46a098465c9b3c4d6e68962fb8', 'request_id': 'req-64a12d36-3e22-410c-950a-aa5935773752'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.865459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1333/2622] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:33 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:33.866725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7f943-b0dd-4f0d-a467-fb1bc9113273 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:33.911996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] The token cache is being invalidate because project 8fdfac46a098465c9b3c4d6e68962fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:33.912814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.915361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:33.919471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1290/2623] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:33 2022] DELETE /identity/v3/projects/8fdfac46a098465c9b3c4d6e68962fb8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:33.924403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:33.933187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:34.081083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7f943-b0dd-4f0d-a467-fb1bc9113273 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:34.093241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cb0f8-3b60-45cd-9da5-17e05f37811b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:34.094310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cb0f8-3b60-45cd-9da5-17e05f37811b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:34.095657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cb0f8-3b60-45cd-9da5-17e05f37811b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:34.095657 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cb0f8-3b60-45cd-9da5-17e05f37811b admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:34.096793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cb0f8-3b60-45cd-9da5-17e05f37811b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:34.100896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cb0f8-3b60-45cd-9da5-17e05f37811b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:34.138633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:34.154689 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a12d36-3e22-410c-950a-aa5935773752 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:34.156074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1334/2624] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:33 2022] POST /identity/v3/users => generated 294 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:34.160427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cb0f8-3b60-45cd-9da5-17e05f37811b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:34.165259 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:34.167024 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:34.167495 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:34.167846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-768cb0f8-3b60-45cd-9da5-17e05f37811b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:34.168041 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] PATH_INFO: `/v3/users/63b6c81f274542f2b4843bf16e1e0f88` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:34.170222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b6c81f274542f2b4843bf16e1e0f88)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:34.173991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:34.206013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cb0f8-3b60-45cd-9da5-17e05f37811b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:34.219106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7140280-5956-48cf-9286-3ed1f4b10d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:34.220211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7140280-5956-48cf-9286-3ed1f4b10d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:34.220897 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7140280-5956-48cf-9286-3ed1f4b10d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:34.221346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7140280-5956-48cf-9286-3ed1f4b10d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:34.222822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7140280-5956-48cf-9286-3ed1f4b10d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:34.225461 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7140280-5956-48cf-9286-3ed1f4b10d0d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:34.252794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1335/2625] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:34.259813 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7140280-5956-48cf-9286-3ed1f4b10d0d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:34.271440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7140280-5956-48cf-9286-3ed1f4b10d0d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:34.279471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e9722-b3e5-46de-aa89-96afee6664ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:34.280983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e9722-b3e5-46de-aa89-96afee6664ff admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:34.281281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e9722-b3e5-46de-aa89-96afee6664ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:34.281680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e9722-b3e5-46de-aa89-96afee6664ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:34.282538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e9722-b3e5-46de-aa89-96afee6664ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:34.299156 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b6c81f274542f2b4843bf16e1e0f88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:34.302263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e9722-b3e5-46de-aa89-96afee6664ff admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:34.328821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b6c81f274542f2b4843bf16e1e0f88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:34.332839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b6c81f274542f2b4843bf16e1e0f88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:34.358350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1336/2626] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:34.360230 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e9722-b3e5-46de-aa89-96afee6664ff admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:34.387310 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b6c81f274542f2b4843bf16e1e0f88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:34.389899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3de2fdda-19f7-4676-bc78-a22e5d595efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b6c81f274542f2b4843bf16e1e0f88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:34.395603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e9722-b3e5-46de-aa89-96afee6664ff admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:34.413578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07449325-bd37-4a0d-8faf-90476ac485ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:34.422879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07449325-bd37-4a0d-8faf-90476ac485ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:34.423346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07449325-bd37-4a0d-8faf-90476ac485ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:34.424176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1291/2627] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:33 2022] DELETE /identity/v3/users/63b6c81f274542f2b4843bf16e1e0f88 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:34.426774 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07449325-bd37-4a0d-8faf-90476ac485ae admin admin] PATH_INFO: `/v3/projects/8deadfc0eb0f4361ac4d6237ef41e262/users/edbed5d693bf4d7387d41dcfa4221455/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:34.463958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07449325-bd37-4a0d-8faf-90476ac485ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8deadfc0eb0f4361ac4d6237ef41e262, user_id=edbed5d693bf4d7387d41dcfa4221455, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:34.472393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07449325-bd37-4a0d-8faf-90476ac485ae admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:34.535161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1337/2628] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:34 2022] PUT /identity/v3/projects/8deadfc0eb0f4361ac4d6237ef41e262/users/edbed5d693bf4d7387d41dcfa4221455/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:34.559129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79d028-cc5c-4e79-ac1a-acaa2309c2f8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:34.559603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79d028-cc5c-4e79-ac1a-acaa2309c2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:34.560071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79d028-cc5c-4e79-ac1a-acaa2309c2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:34.649499 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7c79d028-cc5c-4e79-ac1a-acaa2309c2f8 None None] MFA Rules not processed for user `edbed5d693bf4d7387d41dcfa4221455`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:34.652700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7c79d028-cc5c-4e79-ac1a-acaa2309c2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:34.854373 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:34.907399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1292/2629] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:34.922695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07449325-bd37-4a0d-8faf-90476ac485ae admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:34.932107 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-07449325-bd37-4a0d-8faf-90476ac485ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:35.300537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07449325-bd37-4a0d-8faf-90476ac485ae admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:35.330143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41c705-db91-41a6-913e-6db58108e52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:35.331381 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41c705-db91-41a6-913e-6db58108e52d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:35.331914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41c705-db91-41a6-913e-6db58108e52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:35.332409 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41c705-db91-41a6-913e-6db58108e52d admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:35.337321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41c705-db91-41a6-913e-6db58108e52d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:35.348536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41c705-db91-41a6-913e-6db58108e52d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:35.447786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1338/2630] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:34 2022] POST /identity/v3/projects => generated 385 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:35.452353 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79d028-cc5c-4e79-ac1a-acaa2309c2f8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:35.471127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79d028-cc5c-4e79-ac1a-acaa2309c2f8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:35.575276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41c705-db91-41a6-913e-6db58108e52d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:35.580272 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cf41c705-db91-41a6-913e-6db58108e52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:35.651268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc6150-9263-4a73-8216-a19334ce0a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:35.651984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc6150-9263-4a73-8216-a19334ce0a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:35.652267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc6150-9263-4a73-8216-a19334ce0a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:35.652541 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc6150-9263-4a73-8216-a19334ce0a14 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:35.653274 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc6150-9263-4a73-8216-a19334ce0a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:35.657128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc6150-9263-4a73-8216-a19334ce0a14 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:35.725029 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1293/2631] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:35 2022] POST /identity/v3/users => generated 299 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:35.728807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc6150-9263-4a73-8216-a19334ce0a14 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:35.759975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc6150-9263-4a73-8216-a19334ce0a14 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:35.766176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab28f39-96a7-40e2-9536-d8beef3a6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:35.767287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab28f39-96a7-40e2-9536-d8beef3a6195 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:35.767745 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab28f39-96a7-40e2-9536-d8beef3a6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:35.768582 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab28f39-96a7-40e2-9536-d8beef3a6195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:35.769397 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab28f39-96a7-40e2-9536-d8beef3a6195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:35.771430 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab28f39-96a7-40e2-9536-d8beef3a6195 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:35.800143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1294/2632] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:35.810411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab28f39-96a7-40e2-9536-d8beef3a6195 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:35.825164 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41c705-db91-41a6-913e-6db58108e52d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:35.833426 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab28f39-96a7-40e2-9536-d8beef3a6195 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:35.834722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:35.836573 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:35.837202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:35.837944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] PATH_INFO: `/v3/projects/b42aedc3113e493e9d4ab266aec24e3a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:35.841482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42aedc3113e493e9d4ab266aec24e3a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:35.845683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:35.846684 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aca242-a11a-4539-ade8-bc9f6b1233b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:35.847649 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aca242-a11a-4539-ade8-bc9f6b1233b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:35.847932 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aca242-a11a-4539-ade8-bc9f6b1233b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:35.848511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aca242-a11a-4539-ade8-bc9f6b1233b4 admin admin] PATH_INFO: `/v3/projects/6b6f5e2d7da847508f958cbd400ca560/users/4856d2f9acb1487f8e8519c3ea7fc673/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:35.876189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aca242-a11a-4539-ade8-bc9f6b1233b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6f5e2d7da847508f958cbd400ca560, user_id=4856d2f9acb1487f8e8519c3ea7fc673, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:35.879452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aca242-a11a-4539-ade8-bc9f6b1233b4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:35.906081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1295/2633] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:35 2022] PUT /identity/v3/projects/6b6f5e2d7da847508f958cbd400ca560/users/4856d2f9acb1487f8e8519c3ea7fc673/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:35.912537 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aca242-a11a-4539-ade8-bc9f6b1233b4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:35.943814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:35.972006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42aedc3113e493e9d4ab266aec24e3a', 'request_id': 'req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:35.994342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42aedc3113e493e9d4ab266aec24e3a', 'request_id': 'req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:36.010105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42aedc3113e493e9d4ab266aec24e3a', 'request_id': 'req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:36.032911 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] The token cache is being invalidate because project b42aedc3113e493e9d4ab266aec24e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:36.033383 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:36.034655 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5703eb30-639f-4b07-a18e-fd3d70bf8d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:36.037343 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1339/2634] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:35 2022] DELETE /identity/v3/projects/b42aedc3113e493e9d4ab266aec24e3a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:36.042534 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aca242-a11a-4539-ade8-bc9f6b1233b4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:36.048465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9255f6-93cd-435b-a0ac-c3c54e77439f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:36.049277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9255f6-93cd-435b-a0ac-c3c54e77439f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:36.049979 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9255f6-93cd-435b-a0ac-c3c54e77439f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:36.079157 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9f9255f6-93cd-435b-a0ac-c3c54e77439f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:36.080863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9f9255f6-93cd-435b-a0ac-c3c54e77439f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:36.154450 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:36.178580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec341c9-6a22-431a-82d0-31229c02acd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:36.180078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec341c9-6a22-431a-82d0-31229c02acd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:36.180544 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec341c9-6a22-431a-82d0-31229c02acd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:36.180996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec341c9-6a22-431a-82d0-31229c02acd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:36.182496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec341c9-6a22-431a-82d0-31229c02acd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:36.184396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec341c9-6a22-431a-82d0-31229c02acd7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:36.199700 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1296/2635] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:36.204955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec341c9-6a22-431a-82d0-31229c02acd7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:36.207877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-aec341c9-6a22-431a-82d0-31229c02acd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:36.223023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec341c9-6a22-431a-82d0-31229c02acd7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:36.232458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860229a-1058-41b8-89f5-4784faafa0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:36.233221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860229a-1058-41b8-89f5-4784faafa0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:36.233678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860229a-1058-41b8-89f5-4784faafa0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:36.234351 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860229a-1058-41b8-89f5-4784faafa0b7 admin admin] PATH_INFO: `/v3/projects/6b6f5e2d7da847508f958cbd400ca560/users/4856d2f9acb1487f8e8519c3ea7fc673/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:36.236248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1340/2636] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:36.242581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860229a-1058-41b8-89f5-4784faafa0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6f5e2d7da847508f958cbd400ca560, user_id=4856d2f9acb1487f8e8519c3ea7fc673, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:36.242581 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860229a-1058-41b8-89f5-4784faafa0b7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:36.251749 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb80db1-2ded-4be3-b367-1af5cf2ab22a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:36.252660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb80db1-2ded-4be3-b367-1af5cf2ab22a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:36.253637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb80db1-2ded-4be3-b367-1af5cf2ab22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:36.265889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1297/2637] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:36 2022] PUT /identity/v3/projects/6b6f5e2d7da847508f958cbd400ca560/users/4856d2f9acb1487f8e8519c3ea7fc673/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:36.272874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64c23-4493-41fd-8363-cc6256325b51 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:36.274481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64c23-4493-41fd-8363-cc6256325b51 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:36.277224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64c23-4493-41fd-8363-cc6256325b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:36.294387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8fb80db1-2ded-4be3-b367-1af5cf2ab22a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:36.299234 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8fb80db1-2ded-4be3-b367-1af5cf2ab22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:36.361448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b1d64c23-4493-41fd-8363-cc6256325b51 None None] MFA Rules not processed for user `4856d2f9acb1487f8e8519c3ea7fc673`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:36.363117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b1d64c23-4493-41fd-8363-cc6256325b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:36.530965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1341/2638] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:36.536277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb80db1-2ded-4be3-b367-1af5cf2ab22a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:36.562406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb80db1-2ded-4be3-b367-1af5cf2ab22a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:36.577291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:36.578842 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:36.579732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:36.581595 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1298/2639] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:36.587021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:36.588082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:36.588638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:36.590358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:36.610700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1342/2640] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:36.616725 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64c23-4493-41fd-8363-cc6256325b51 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:36.635131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64c23-4493-41fd-8363-cc6256325b51 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:36.639010 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:36.642921 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:36.651113 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d8959-2fdf-4e13-b8d2-daf3df79a67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:36.652264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d8959-2fdf-4e13-b8d2-daf3df79a67c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:36.652622 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d8959-2fdf-4e13-b8d2-daf3df79a67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:36.652794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d8959-2fdf-4e13-b8d2-daf3df79a67c admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:36.653530 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d8959-2fdf-4e13-b8d2-daf3df79a67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:36.666050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d8959-2fdf-4e13-b8d2-daf3df79a67c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:36.757116 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1299/2641] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:36 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:36.767474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d8959-2fdf-4e13-b8d2-daf3df79a67c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:36.844298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1061cbe-4a30-4c56-96b9-b8b8012a62f2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:36.877923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:37.016004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d8959-2fdf-4e13-b8d2-daf3df79a67c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:37.022356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb802ec-7867-4e9c-8546-7edc0c8fa945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:37.023161 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb802ec-7867-4e9c-8546-7edc0c8fa945 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:37.024490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb802ec-7867-4e9c-8546-7edc0c8fa945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:37.024997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb802ec-7867-4e9c-8546-7edc0c8fa945 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:37.025847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb802ec-7867-4e9c-8546-7edc0c8fa945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:37.028790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b56574-f4f4-496f-bef1-d65cd41b35f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:37.030427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb802ec-7867-4e9c-8546-7edc0c8fa945 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:37.032396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b56574-f4f4-496f-bef1-d65cd41b35f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:37.032396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b56574-f4f4-496f-bef1-d65cd41b35f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:37.032942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b56574-f4f4-496f-bef1-d65cd41b35f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:37.109563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1300/2642] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:36 2022] POST /identity/v3/users => generated 292 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:37.116415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb802ec-7867-4e9c-8546-7edc0c8fa945 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:37.135284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb802ec-7867-4e9c-8546-7edc0c8fa945 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:37.142179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d755f1-7e57-4d7b-b9a2-674e8b854705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:37.142933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d755f1-7e57-4d7b-b9a2-674e8b854705 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:37.143221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d755f1-7e57-4d7b-b9a2-674e8b854705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:37.143501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d755f1-7e57-4d7b-b9a2-674e8b854705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:37.144262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d755f1-7e57-4d7b-b9a2-674e8b854705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:37.146309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d755f1-7e57-4d7b-b9a2-674e8b854705 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:37.184484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1301/2643] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:37.189012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d755f1-7e57-4d7b-b9a2-674e8b854705 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:37.218225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d755f1-7e57-4d7b-b9a2-674e8b854705 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:37.228803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b3885-76ac-4469-a597-cf557e5f0bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:37.231035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b56574-f4f4-496f-bef1-d65cd41b35f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:37.232553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b3885-76ac-4469-a597-cf557e5f0bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:37.232806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b56574-f4f4-496f-bef1-d65cd41b35f6 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:37.233221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b3885-76ac-4469-a597-cf557e5f0bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:37.233825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b3885-76ac-4469-a597-cf557e5f0bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:37.235273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b3885-76ac-4469-a597-cf557e5f0bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:37.238338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b3885-76ac-4469-a597-cf557e5f0bc5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:37.258595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1343/2644] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:37.266148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1302/2645] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:37.267449 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b56574-f4f4-496f-bef1-d65cd41b35f6 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:37.293913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b3885-76ac-4469-a597-cf557e5f0bc5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:37.294967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b56574-f4f4-496f-bef1-d65cd41b35f6 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:37.305179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667811c-7ccd-4f08-bb19-fb375d8d0fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:37.307571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667811c-7ccd-4f08-bb19-fb375d8d0fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:37.307810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667811c-7ccd-4f08-bb19-fb375d8d0fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:37.308305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667811c-7ccd-4f08-bb19-fb375d8d0fe7 admin admin] PATH_INFO: `/v3/projects/5bb093666fae4bd6a6f7c1294c3910c8/users/1a6b701a901043f99a59710766fa1a98/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:37.317571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:37.330827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b3885-76ac-4469-a597-cf557e5f0bc5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:37.354951 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:37.356827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:37.356827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:37.357287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] PATH_INFO: `/v3/users/edbed5d693bf4d7387d41dcfa4221455` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:37.358904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edbed5d693bf4d7387d41dcfa4221455)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:37.362112 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:37.382683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667811c-7ccd-4f08-bb19-fb375d8d0fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb093666fae4bd6a6f7c1294c3910c8, user_id=1a6b701a901043f99a59710766fa1a98, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:37.391143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667811c-7ccd-4f08-bb19-fb375d8d0fe7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:37.431031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1344/2646] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:37 2022] PUT /identity/v3/projects/5bb093666fae4bd6a6f7c1294c3910c8/users/1a6b701a901043f99a59710766fa1a98/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:37.446936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddebd4c-1259-4e93-8816-3d0f75cdccf3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:37.447236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddebd4c-1259-4e93-8816-3d0f75cdccf3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:37.449167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddebd4c-1259-4e93-8816-3d0f75cdccf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:37.522432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1ddebd4c-1259-4e93-8816-3d0f75cdccf3 None None] MFA Rules not processed for user `1a6b701a901043f99a59710766fa1a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:37.531626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1ddebd4c-1259-4e93-8816-3d0f75cdccf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:37.560981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edbed5d693bf4d7387d41dcfa4221455'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:37.602282 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edbed5d693bf4d7387d41dcfa4221455'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:37.610091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edbed5d693bf4d7387d41dcfa4221455'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:37.709593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edbed5d693bf4d7387d41dcfa4221455'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:37.714576 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-558219dc-316a-4398-bc3d-ebd40e0cd026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edbed5d693bf4d7387d41dcfa4221455'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:37.762390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1303/2647] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:37 2022] DELETE /identity/v3/users/edbed5d693bf4d7387d41dcfa4221455 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:37.771181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d5df5-d826-4de7-bda3-2049d22b6706 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:37.772144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d5df5-d826-4de7-bda3-2049d22b6706 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:37.773280 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d5df5-d826-4de7-bda3-2049d22b6706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:37.809627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-738d5df5-d826-4de7-bda3-2049d22b6706 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:37.812862 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-738d5df5-d826-4de7-bda3-2049d22b6706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:37.971790 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1345/2648] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 527 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:37.995788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddebd4c-1259-4e93-8816-3d0f75cdccf3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:37.997844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1ddebd4c-1259-4e93-8816-3d0f75cdccf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:38.014825 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1304/2649] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:38.063694 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d5df5-d826-4de7-bda3-2049d22b6706 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:38.067711 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-738d5df5-d826-4de7-bda3-2049d22b6706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:38.102539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:38.204139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d5df5-d826-4de7-bda3-2049d22b6706 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:38.210498 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee0c53-0ab9-4efd-a386-341971e79414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:38.212344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee0c53-0ab9-4efd-a386-341971e79414 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:38.213799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee0c53-0ab9-4efd-a386-341971e79414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:38.214810 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee0c53-0ab9-4efd-a386-341971e79414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:38.240344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee0c53-0ab9-4efd-a386-341971e79414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:38.242875 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddebd4c-1259-4e93-8816-3d0f75cdccf3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:38.249062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee0c53-0ab9-4efd-a386-341971e79414 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:38.270147 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1305/2650] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:38.424507 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dded38f-4a3a-4b25-b191-293164be7475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:38.429076 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded38f-4a3a-4b25-b191-293164be7475 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:38.430618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded38f-4a3a-4b25-b191-293164be7475 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:38.431518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded38f-4a3a-4b25-b191-293164be7475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:38.622835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dded38f-4a3a-4b25-b191-293164be7475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:38.624465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dded38f-4a3a-4b25-b191-293164be7475 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:38.664442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1346/2651] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:30:38.677507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee0c53-0ab9-4efd-a386-341971e79414 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:38.679831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a7ee0c53-0ab9-4efd-a386-341971e79414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:38.705388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee0c53-0ab9-4efd-a386-341971e79414 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:38.722835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:38.724203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:38.724203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:38.724203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] PATH_INFO: `/v3/users/1a6b701a901043f99a59710766fa1a98` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:38.726819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a6b701a901043f99a59710766fa1a98)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:38.727996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:38.765127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:38.831927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a6b701a901043f99a59710766fa1a98'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:38.843078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a6b701a901043f99a59710766fa1a98'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:38.844042 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a6b701a901043f99a59710766fa1a98'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:38.862838 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a6b701a901043f99a59710766fa1a98'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:38.863898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a6b701a901043f99a59710766fa1a98'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:38.887644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1306/2652] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:38 2022] DELETE /identity/v3/users/1a6b701a901043f99a59710766fa1a98 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:38.892618 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f51c0-19b5-4506-999e-0218dfeabc10 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:38.892919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f51c0-19b5-4506-999e-0218dfeabc10 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:38.893209 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f51c0-19b5-4506-999e-0218dfeabc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:38.941248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-138f51c0-19b5-4506-999e-0218dfeabc10 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:38.947246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-138f51c0-19b5-4506-999e-0218dfeabc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:39.113032 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1347/2653] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:39.143906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:39.146216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:39.262625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f831a-9f27-440a-8bcd-2ca5fc242340 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:39.267830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69684409-74d2-4a31-bcda-2235251b5f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:39.268575 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69684409-74d2-4a31-bcda-2235251b5f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:39.268867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69684409-74d2-4a31-bcda-2235251b5f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:39.269151 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69684409-74d2-4a31-bcda-2235251b5f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:39.284402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69684409-74d2-4a31-bcda-2235251b5f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:39.286011 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69684409-74d2-4a31-bcda-2235251b5f25 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:39.304545 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1307/2654] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:39.446967 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f51c0-19b5-4506-999e-0218dfeabc10 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:39.457799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-138f51c0-19b5-4506-999e-0218dfeabc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:39.498978 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f51c0-19b5-4506-999e-0218dfeabc10 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:39.532422 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:39.535353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:39.535780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:39.537090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] PATH_INFO: `/v3/projects/8deadfc0eb0f4361ac4d6237ef41e262` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:39.541597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8deadfc0eb0f4361ac4d6237ef41e262)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:39.543902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:39.671798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:39.702193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8deadfc0eb0f4361ac4d6237ef41e262', 'request_id': 'req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:39.721765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8deadfc0eb0f4361ac4d6237ef41e262', 'request_id': 'req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:39.735405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8deadfc0eb0f4361ac4d6237ef41e262', 'request_id': 'req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:39.763256 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] The token cache is being invalidate because project 8deadfc0eb0f4361ac4d6237ef41e262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:39.763555 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:39.766471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:39.780465 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1348/2655] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:39 2022] DELETE /identity/v3/projects/8deadfc0eb0f4361ac4d6237ef41e262 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:39.787475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69684409-74d2-4a31-bcda-2235251b5f25 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:39.827731 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69684409-74d2-4a31-bcda-2235251b5f25 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:40.027148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:40.028069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:40.028543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:40.029154 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] PATH_INFO: `/v3/users/4856d2f9acb1487f8e8519c3ea7fc673` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:40.030843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4856d2f9acb1487f8e8519c3ea7fc673)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:40.033089 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:40.160598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:40.227859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4856d2f9acb1487f8e8519c3ea7fc673'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.238761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4856d2f9acb1487f8e8519c3ea7fc673'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.239789 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4856d2f9acb1487f8e8519c3ea7fc673'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.249679 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:40.251194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:40.261437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4856d2f9acb1487f8e8519c3ea7fc673'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.262335 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-40d72a16-b4be-4c5b-a49b-e428fcd2840f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4856d2f9acb1487f8e8519c3ea7fc673'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.285603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4159f-a8a7-43ee-8c5f-b6aa4cd53cce admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:40.290044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1308/2656] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:39 2022] DELETE /identity/v3/users/4856d2f9acb1487f8e8519c3ea7fc673 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:40.452540 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:40.454683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:40.455599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:40.456501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] PATH_INFO: `/v3/projects/5bb093666fae4bd6a6f7c1294c3910c8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:40.459402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb093666fae4bd6a6f7c1294c3910c8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:40.465630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:40.560468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb093666fae4bd6a6f7c1294c3910c8', 'request_id': 'req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.584045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb093666fae4bd6a6f7c1294c3910c8', 'request_id': 'req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.596230 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb093666fae4bd6a6f7c1294c3910c8', 'request_id': 'req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.613202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] The token cache is being invalidate because project 5bb093666fae4bd6a6f7c1294c3910c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:40.613522 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.614347 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0621d0c-2b72-4107-a4ee-b7a6ee536c29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:40.617764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1349/2657] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:40 2022] DELETE /identity/v3/projects/5bb093666fae4bd6a6f7c1294c3910c8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:40.622236 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954888c-7b89-4f20-b73d-0306cfa01172 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:40.623506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954888c-7b89-4f20-b73d-0306cfa01172 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:40.623972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954888c-7b89-4f20-b73d-0306cfa01172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:40.670283 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2954888c-7b89-4f20-b73d-0306cfa01172 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:40.672015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2954888c-7b89-4f20-b73d-0306cfa01172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:40.891533 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1309/2658] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:40.905367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd8bc4-e899-43d2-b5cc-6497ea19b653 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:40.906735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd8bc4-e899-43d2-b5cc-6497ea19b653 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:40.906735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd8bc4-e899-43d2-b5cc-6497ea19b653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:40.964072 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-71fd8bc4-e899-43d2-b5cc-6497ea19b653 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:40.969116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-71fd8bc4-e899-43d2-b5cc-6497ea19b653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:41.002770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1350/2659] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:41.003437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954888c-7b89-4f20-b73d-0306cfa01172 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:41.031460 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954888c-7b89-4f20-b73d-0306cfa01172 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:41.040131 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:41.043835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:41.044502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:41.044934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:41.046324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:41.047458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:41.049475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:41.082326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1310/2660] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:41.084098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd8bc4-e899-43d2-b5cc-6497ea19b653 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:41.107242 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd8bc4-e899-43d2-b5cc-6497ea19b653 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:41.114202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b298b0-a72b-4eb9-8c75-794ff7998234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:41.115035 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b298b0-a72b-4eb9-8c75-794ff7998234 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:41.115325 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b298b0-a72b-4eb9-8c75-794ff7998234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:41.115609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b298b0-a72b-4eb9-8c75-794ff7998234 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:41.116329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b298b0-a72b-4eb9-8c75-794ff7998234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:41.119268 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b298b0-a72b-4eb9-8c75-794ff7998234 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:41.197778 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1351/2661] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:41 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:41.200766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:41.202199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b298b0-a72b-4eb9-8c75-794ff7998234 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:41.203552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:41.288405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230372c8-3c37-4339-a54a-e14e1854a101 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:41.306279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:41.443697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b298b0-a72b-4eb9-8c75-794ff7998234 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:41.468128 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d18c0-2c34-4e0f-81dd-bc0af4d26320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:41.469248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d18c0-2c34-4e0f-81dd-bc0af4d26320 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:41.469902 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d18c0-2c34-4e0f-81dd-bc0af4d26320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:41.470401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d18c0-2c34-4e0f-81dd-bc0af4d26320 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:41.473388 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d18c0-2c34-4e0f-81dd-bc0af4d26320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:41.486491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d18c0-2c34-4e0f-81dd-bc0af4d26320 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:41.572168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1352/2662] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:41 2022] POST /identity/v3/users => generated 292 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:41.578680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d18c0-2c34-4e0f-81dd-bc0af4d26320 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:41.601220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:41.604006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d18c0-2c34-4e0f-81dd-bc0af4d26320 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:41.604264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:41.604802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:41.605561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] PATH_INFO: `/v3/projects/6b6f5e2d7da847508f958cbd400ca560` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:41.609013 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b6f5e2d7da847508f958cbd400ca560)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:41.613225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:41.621753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe867560-ded8-4ecf-b815-c6189128a063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:41.624095 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe867560-ded8-4ecf-b815-c6189128a063 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:41.624592 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe867560-ded8-4ecf-b815-c6189128a063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:41.625097 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe867560-ded8-4ecf-b815-c6189128a063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:41.628816 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe867560-ded8-4ecf-b815-c6189128a063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:41.637649 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe867560-ded8-4ecf-b815-c6189128a063 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:41.690475 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1353/2663] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:41.699149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe867560-ded8-4ecf-b815-c6189128a063 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:41.744955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe867560-ded8-4ecf-b815-c6189128a063 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:41.767336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166551b4-78ea-41d9-91e5-e52d5694952e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:41.768378 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166551b4-78ea-41d9-91e5-e52d5694952e admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:41.768880 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166551b4-78ea-41d9-91e5-e52d5694952e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:41.772286 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166551b4-78ea-41d9-91e5-e52d5694952e admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:41.777826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166551b4-78ea-41d9-91e5-e52d5694952e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:41.782688 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166551b4-78ea-41d9-91e5-e52d5694952e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:41.809211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:41.822368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b6f5e2d7da847508f958cbd400ca560', 'request_id': 'req-939c1113-bcf8-4f10-a5ac-3832c604f948'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:41.834684 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1354/2664] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:41.840927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166551b4-78ea-41d9-91e5-e52d5694952e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:41.841294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b6f5e2d7da847508f958cbd400ca560', 'request_id': 'req-939c1113-bcf8-4f10-a5ac-3832c604f948'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:41.855827 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b6f5e2d7da847508f958cbd400ca560', 'request_id': 'req-939c1113-bcf8-4f10-a5ac-3832c604f948'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:41.869266 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166551b4-78ea-41d9-91e5-e52d5694952e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:41.888633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] The token cache is being invalidate because project 6b6f5e2d7da847508f958cbd400ca560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:41.888934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:41.895405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-939c1113-bcf8-4f10-a5ac-3832c604f948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:41.906329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1311/2665] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:41 2022] DELETE /identity/v3/projects/6b6f5e2d7da847508f958cbd400ca560 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:41.908625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9f337-0d93-4c35-8a40-cc6e02784f8c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:41.909756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9f337-0d93-4c35-8a40-cc6e02784f8c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:41.909756 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9f337-0d93-4c35-8a40-cc6e02784f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:41.974781 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7ff9f337-0d93-4c35-8a40-cc6e02784f8c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:41.976128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7ff9f337-0d93-4c35-8a40-cc6e02784f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:42.084588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e45a0e-9528-4cd0-82a7-053d87734b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:42.091696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e45a0e-9528-4cd0-82a7-053d87734b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:42.092090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e45a0e-9528-4cd0-82a7-053d87734b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:42.092441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e45a0e-9528-4cd0-82a7-053d87734b28 admin admin] PATH_INFO: `/v3/projects/0698f89e78e84a19847c1b05bc05311b/users/b572ed434c714fa78885deb417df864c/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:42.173965 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e45a0e-9528-4cd0-82a7-053d87734b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0698f89e78e84a19847c1b05bc05311b, user_id=b572ed434c714fa78885deb417df864c, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:42.182884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1312/2666] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:42.199888 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e45a0e-9528-4cd0-82a7-053d87734b28 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:42.201085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f058de-5718-4265-9653-177f9dddabd2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:42.208484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f058de-5718-4265-9653-177f9dddabd2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:42.212088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f058de-5718-4265-9653-177f9dddabd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:42.319315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1355/2667] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:41 2022] PUT /identity/v3/projects/0698f89e78e84a19847c1b05bc05311b/users/b572ed434c714fa78885deb417df864c/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:42.326430 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04f8de-8576-4c92-abec-6344501428c1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:42.326910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04f8de-8576-4c92-abec-6344501428c1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:42.327406 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04f8de-8576-4c92-abec-6344501428c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:42.349434 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-31f058de-5718-4265-9653-177f9dddabd2 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:42.353737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-31f058de-5718-4265-9653-177f9dddabd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:42.466761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0a04f8de-8576-4c92-abec-6344501428c1 None None] MFA Rules not processed for user `b572ed434c714fa78885deb417df864c`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:42.472740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0a04f8de-8576-4c92-abec-6344501428c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:42.753653 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1313/2668] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 557 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:42.756327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f058de-5718-4265-9653-177f9dddabd2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:42.797855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f058de-5718-4265-9653-177f9dddabd2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:42.810607 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17018-4183-4bc4-b5b1-3258d2db9fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:42.811390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17018-4183-4bc4-b5b1-3258d2db9fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:42.811690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17018-4183-4bc4-b5b1-3258d2db9fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:42.812012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17018-4183-4bc4-b5b1-3258d2db9fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:42.812883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef17018-4183-4bc4-b5b1-3258d2db9fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:42.813194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef17018-4183-4bc4-b5b1-3258d2db9fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:42.814875 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef17018-4183-4bc4-b5b1-3258d2db9fd0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:42.895320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1356/2669] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 575 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:42.916048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1314/2670] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:42.918315 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04f8de-8576-4c92-abec-6344501428c1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:42.922309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17018-4183-4bc4-b5b1-3258d2db9fd0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:42.922569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0a04f8de-8576-4c92-abec-6344501428c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:42.928486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:42.941277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17018-4183-4bc4-b5b1-3258d2db9fd0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:42.947755 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046ab94-3593-4286-9116-541857691df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:42.948730 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ab94-3593-4286-9116-541857691df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:42.949180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ab94-3593-4286-9116-541857691df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:42.949620 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ab94-3593-4286-9116-541857691df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:42.950688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046ab94-3593-4286-9116-541857691df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:42.952842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046ab94-3593-4286-9116-541857691df5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:43.001286 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1315/2671] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:42 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:43.007129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046ab94-3593-4286-9116-541857691df5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:43.114698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:43.158576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04f8de-8576-4c92-abec-6344501428c1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:43.173808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046ab94-3593-4286-9116-541857691df5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:43.186067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb1ade-1618-462d-bf23-46643adb3579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:43.187406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1ade-1618-462d-bf23-46643adb3579 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:43.189835 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1ade-1618-462d-bf23-46643adb3579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:43.190758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb1ade-1618-462d-bf23-46643adb3579 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:43.191604 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb1ade-1618-462d-bf23-46643adb3579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:43.199327 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb1ade-1618-462d-bf23-46643adb3579 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:43.243793 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd934c83-5354-4ee1-8bf7-653082aab8a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:43.244716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd934c83-5354-4ee1-8bf7-653082aab8a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:43.245070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd934c83-5354-4ee1-8bf7-653082aab8a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:43.245420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd934c83-5354-4ee1-8bf7-653082aab8a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:43.279687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1316/2672] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:43 2022] POST /identity/v3/users => generated 287 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:43.287309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb1ade-1618-462d-bf23-46643adb3579 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:43.332427 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb1ade-1618-462d-bf23-46643adb3579 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:43.341476 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7864472-2558-464d-8732-41f331e629a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:43.343329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7864472-2558-464d-8732-41f331e629a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:43.343983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7864472-2558-464d-8732-41f331e629a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:43.344597 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7864472-2558-464d-8732-41f331e629a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:43.346303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7864472-2558-464d-8732-41f331e629a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:43.349944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7864472-2558-464d-8732-41f331e629a3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:43.378958 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1317/2673] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:43.378958 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7864472-2558-464d-8732-41f331e629a3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:43.381523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a7864472-2558-464d-8732-41f331e629a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:43.406407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd934c83-5354-4ee1-8bf7-653082aab8a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:43.408944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd934c83-5354-4ee1-8bf7-653082aab8a8 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:43.417372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7864472-2558-464d-8732-41f331e629a3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:43.422929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1357/2674] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:43.424317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd934c83-5354-4ee1-8bf7-653082aab8a8 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:43.432481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:43.440263 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:43.440641 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:43.440926 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] PATH_INFO: `/v3/users/405ac2f6c07545efaa818e10fc788c05` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:43.442702 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405ac2f6c07545efaa818e10fc788c05)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:43.446212 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:43.460801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd934c83-5354-4ee1-8bf7-653082aab8a8 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:43.475506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38518c-0328-421f-8521-37f9172e2b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:43.478660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38518c-0328-421f-8521-37f9172e2b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:43.479537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38518c-0328-421f-8521-37f9172e2b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:43.481950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38518c-0328-421f-8521-37f9172e2b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:43.485566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38518c-0328-421f-8521-37f9172e2b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:43.491150 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38518c-0328-421f-8521-37f9172e2b6f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:43.518840 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405ac2f6c07545efaa818e10fc788c05'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:43.530119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405ac2f6c07545efaa818e10fc788c05'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:43.531297 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405ac2f6c07545efaa818e10fc788c05'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:43.543794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1358/2675] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:43.548999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38518c-0328-421f-8521-37f9172e2b6f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:43.550793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405ac2f6c07545efaa818e10fc788c05'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:43.551573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4853241b-6d66-41ae-9d35-58acb572a3b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405ac2f6c07545efaa818e10fc788c05'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:43.574419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1318/2676] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:43 2022] DELETE /identity/v3/users/405ac2f6c07545efaa818e10fc788c05 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:43.576817 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38518c-0328-421f-8521-37f9172e2b6f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:43.701439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70188d-f0ef-420a-9765-0cb6b5b32480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:43.702486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70188d-f0ef-420a-9765-0cb6b5b32480 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:43.702997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70188d-f0ef-420a-9765-0cb6b5b32480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:43.703335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70188d-f0ef-420a-9765-0cb6b5b32480 admin admin] PATH_INFO: `/v3/projects/4194834314a54388a880cbe00c43a79f/users/9d078f75a97a478d91bd101f637b9adf/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:43.708786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:43.736114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70188d-f0ef-420a-9765-0cb6b5b32480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4194834314a54388a880cbe00c43a79f, user_id=9d078f75a97a478d91bd101f637b9adf, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:43.743453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70188d-f0ef-420a-9765-0cb6b5b32480 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:43.775317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1359/2677] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:43 2022] PUT /identity/v3/projects/4194834314a54388a880cbe00c43a79f/users/9d078f75a97a478d91bd101f637b9adf/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:43.789556 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d606efb-9003-4384-a5c7-6b7291289bcb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:43.791969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d606efb-9003-4384-a5c7-6b7291289bcb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:43.793843 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d606efb-9003-4384-a5c7-6b7291289bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:43.862484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7d606efb-9003-4384-a5c7-6b7291289bcb None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:43.865536 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7d606efb-9003-4384-a5c7-6b7291289bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:44.066869 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1319/2678] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:44.075459 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70188d-f0ef-420a-9765-0cb6b5b32480 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:44.078068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bc70188d-f0ef-420a-9765-0cb6b5b32480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:44.174261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d606efb-9003-4384-a5c7-6b7291289bcb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:44.175916 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7d606efb-9003-4384-a5c7-6b7291289bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:44.190184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d606efb-9003-4384-a5c7-6b7291289bcb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:44.197334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:44.198754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:44.199119 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:44.199482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] PATH_INFO: `/v3/users/791369c0e2574cdf8047a5f178a3c6db` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:44.201414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] RBAC: Authorizing `identity:delete_user(user_id=791369c0e2574cdf8047a5f178a3c6db)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:44.203697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:44.236876 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70188d-f0ef-420a-9765-0cb6b5b32480 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:44.244812 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e88401-4f61-471e-8845-21a576f031a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:44.245599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e88401-4f61-471e-8845-21a576f031a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:44.245993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e88401-4f61-471e-8845-21a576f031a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:44.246327 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e88401-4f61-471e-8845-21a576f031a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:44.255128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=83,keystone:INSERT=4,keystone:DELETE=13 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:44.264537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e88401-4f61-471e-8845-21a576f031a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:44.266759 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e88401-4f61-471e-8845-21a576f031a3 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:44.288719 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1360/2679] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:44.291968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e88401-4f61-471e-8845-21a576f031a3 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:44.291968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d3e88401-4f61-471e-8845-21a576f031a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:44.310691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e88401-4f61-471e-8845-21a576f031a3 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:44.313792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791369c0e2574cdf8047a5f178a3c6db'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.326125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791369c0e2574cdf8047a5f178a3c6db'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.328138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791369c0e2574cdf8047a5f178a3c6db'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.329579 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:44.331277 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:44.331725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:44.332472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] PATH_INFO: `/v3/projects/52da965db3654006abddd5b40b0905c2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:44.336453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] RBAC: Authorizing `identity:delete_project(project_id=52da965db3654006abddd5b40b0905c2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:44.340087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:44.360124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791369c0e2574cdf8047a5f178a3c6db'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.361295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c8c5203-5607-4a0c-b645-d533e984622c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791369c0e2574cdf8047a5f178a3c6db'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.384090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1320/2680] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:44 2022] DELETE /identity/v3/users/791369c0e2574cdf8047a5f178a3c6db => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:44.444200 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52da965db3654006abddd5b40b0905c2', 'request_id': 'req-24f954c3-8c4b-4be2-87d3-27fae929635b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.465810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52da965db3654006abddd5b40b0905c2', 'request_id': 'req-24f954c3-8c4b-4be2-87d3-27fae929635b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.480074 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52da965db3654006abddd5b40b0905c2', 'request_id': 'req-24f954c3-8c4b-4be2-87d3-27fae929635b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.502964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] The token cache is being invalidate because project 52da965db3654006abddd5b40b0905c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:44.503414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.504523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-24f954c3-8c4b-4be2-87d3-27fae929635b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.506834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1361/2681] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:44 2022] DELETE /identity/v3/projects/52da965db3654006abddd5b40b0905c2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:44.513606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ebdda-fa16-4e1a-b3bb-eb9f68840c12 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:44.514187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ebdda-fa16-4e1a-b3bb-eb9f68840c12 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:44.515224 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ebdda-fa16-4e1a-b3bb-eb9f68840c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:44.542800 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-239ebdda-fa16-4e1a-b3bb-eb9f68840c12 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:44.544334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-239ebdda-fa16-4e1a-b3bb-eb9f68840c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:44.657747 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1321/2682] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:44.665255 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574108fc-bd2d-441a-af85-abcf5b9693ab None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:44.665800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574108fc-bd2d-441a-af85-abcf5b9693ab None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:44.666308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574108fc-bd2d-441a-af85-abcf5b9693ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:44.690562 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ebdda-fa16-4e1a-b3bb-eb9f68840c12 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:44.693090 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-239ebdda-fa16-4e1a-b3bb-eb9f68840c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:44.701089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-574108fc-bd2d-441a-af85-abcf5b9693ab None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:44.704151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-574108fc-bd2d-441a-af85-abcf5b9693ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:44.714524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ebdda-fa16-4e1a-b3bb-eb9f68840c12 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:44.719293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:44.720303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:44.720655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:44.720953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1362/2683] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:44.721124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] PATH_INFO: `/v3/projects/d1af47083c644609a4a84b1273a4087f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:44.722643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1af47083c644609a4a84b1273a4087f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:44.724762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:44.727448 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574108fc-bd2d-441a-af85-abcf5b9693ab None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:44.747641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574108fc-bd2d-441a-af85-abcf5b9693ab None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:44.762599 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a1bf8-6185-4fa9-ac90-e3fd98fbaa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:44.763906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a1bf8-6185-4fa9-ac90-e3fd98fbaa30 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:44.764521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a1bf8-6185-4fa9-ac90-e3fd98fbaa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:44.765030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a1bf8-6185-4fa9-ac90-e3fd98fbaa30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:44.766561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a1bf8-6185-4fa9-ac90-e3fd98fbaa30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:44.767129 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a1bf8-6185-4fa9-ac90-e3fd98fbaa30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:44.770401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a1bf8-6185-4fa9-ac90-e3fd98fbaa30 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:44.798230 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1363/2684] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:30:44.798489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1af47083c644609a4a84b1273a4087f', 'request_id': 'req-8b8be9ec-f181-4c45-a331-522dfec18c95'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.800990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a1bf8-6185-4fa9-ac90-e3fd98fbaa30 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:44.807045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1af47083c644609a4a84b1273a4087f', 'request_id': 'req-8b8be9ec-f181-4c45-a331-522dfec18c95'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.817017 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1af47083c644609a4a84b1273a4087f', 'request_id': 'req-8b8be9ec-f181-4c45-a331-522dfec18c95'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.819603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a1bf8-6185-4fa9-ac90-e3fd98fbaa30 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:44.832378 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] The token cache is being invalidate because project d1af47083c644609a4a84b1273a4087f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:44.832831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.834255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:44.837114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1322/2685] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:44 2022] DELETE /identity/v3/projects/d1af47083c644609a4a84b1273a4087f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:44.842293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:44.844593 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:44.880826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:44.925427 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:44.943944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533db4f-adb1-4132-9e1b-470a8804f59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:44.945765 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533db4f-adb1-4132-9e1b-470a8804f59b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:44.946161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533db4f-adb1-4132-9e1b-470a8804f59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:44.946546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533db4f-adb1-4132-9e1b-470a8804f59b admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:44.947418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d533db4f-adb1-4132-9e1b-470a8804f59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:44.949944 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d533db4f-adb1-4132-9e1b-470a8804f59b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:44.969273 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8be9ec-f181-4c45-a331-522dfec18c95 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:44.984872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:44.985652 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:44.987190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:44.987481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] PATH_INFO: `/v3/users/8ca409fc0da648a28077a3b34465326b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:44.990246 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca409fc0da648a28077a3b34465326b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:44.991627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:44.992751 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1364/2686] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:44 2022] POST /identity/v3/projects => generated 403 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:44.993927 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533db4f-adb1-4132-9e1b-470a8804f59b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:44.996412 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d533db4f-adb1-4132-9e1b-470a8804f59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:45.085170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca409fc0da648a28077a3b34465326b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:45.104016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca409fc0da648a28077a3b34465326b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:45.106246 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca409fc0da648a28077a3b34465326b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:45.129575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533db4f-adb1-4132-9e1b-470a8804f59b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:45.141064 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a1b87-880a-4ca5-823a-6f49db054dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:45.142868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a1b87-880a-4ca5-823a-6f49db054dfc admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:45.143374 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a1b87-880a-4ca5-823a-6f49db054dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:45.143939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a1b87-880a-4ca5-823a-6f49db054dfc admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:45.146057 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a1b87-880a-4ca5-823a-6f49db054dfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:45.153262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a1b87-880a-4ca5-823a-6f49db054dfc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:45.183438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca409fc0da648a28077a3b34465326b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:45.186696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca409fc0da648a28077a3b34465326b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:45.208619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1365/2687] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:44 2022] POST /identity/v3/users => generated 302 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:45.216310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a1b87-880a-4ca5-823a-6f49db054dfc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:45.222627 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5d3a1b87-880a-4ca5-823a-6f49db054dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:45.237227 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1323/2688] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:44 2022] DELETE /identity/v3/users/8ca409fc0da648a28077a3b34465326b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:45.257804 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a1b87-880a-4ca5-823a-6f49db054dfc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:45.407571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4591676-652b-4370-84f5-fe8c40d5661c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:45.408822 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4591676-652b-4370-84f5-fe8c40d5661c admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:45.409755 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4591676-652b-4370-84f5-fe8c40d5661c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:45.411092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4591676-652b-4370-84f5-fe8c40d5661c admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:45.413161 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4591676-652b-4370-84f5-fe8c40d5661c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:45.417942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4591676-652b-4370-84f5-fe8c40d5661c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:45.444808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:45.447407 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1366/2689] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:45.473129 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509cce3-f76e-4aea-96f1-84889bb2ffee admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:45.487920 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411d2dc-7a22-4286-9537-d49925816f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:45.490858 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411d2dc-7a22-4286-9537-d49925816f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:45.491511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411d2dc-7a22-4286-9537-d49925816f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:45.492714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411d2dc-7a22-4286-9537-d49925816f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:45.495646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411d2dc-7a22-4286-9537-d49925816f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:45.501414 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411d2dc-7a22-4286-9537-d49925816f98 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:45.521309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec2602-bbda-440f-a5a3-a95e4b8f3ee0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:45.521682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec2602-bbda-440f-a5a3-a95e4b8f3ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:45.522120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec2602-bbda-440f-a5a3-a95e4b8f3ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:45.524927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1324/2690] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:45.534742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411d2dc-7a22-4286-9537-d49925816f98 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:45.557114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411d2dc-7a22-4286-9537-d49925816f98 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:45.566577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aacf81-cf2e-470d-bd18-03ed08743a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:45.568761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aacf81-cf2e-470d-bd18-03ed08743a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:45.569488 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aacf81-cf2e-470d-bd18-03ed08743a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:45.570751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aacf81-cf2e-470d-bd18-03ed08743a59 admin admin] PATH_INFO: `/v3/projects/f1093cc1cad045eca64cb2047cf227f4/users/175c6e4093f34acf97165852c125ab6e/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:45.577082 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-40ec2602-bbda-440f-a5a3-a95e4b8f3ee0 None None] MFA Rules not processed for user `9d078f75a97a478d91bd101f637b9adf`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:45.580899 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-40ec2602-bbda-440f-a5a3-a95e4b8f3ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:45.608448 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aacf81-cf2e-470d-bd18-03ed08743a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1093cc1cad045eca64cb2047cf227f4, user_id=175c6e4093f34acf97165852c125ab6e, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:45.615415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aacf81-cf2e-470d-bd18-03ed08743a59 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:45.645629 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1325/2691] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:45 2022] PUT /identity/v3/projects/f1093cc1cad045eca64cb2047cf227f4/users/175c6e4093f34acf97165852c125ab6e/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:45.652948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b7e9-556f-4918-9b4c-8d8cd5f59d4f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:45.653614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b7e9-556f-4918-9b4c-8d8cd5f59d4f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:45.655552 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b7e9-556f-4918-9b4c-8d8cd5f59d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:45.702354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6457b7e9-556f-4918-9b4c-8d8cd5f59d4f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:45.705715 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6457b7e9-556f-4918-9b4c-8d8cd5f59d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:45.733766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:45.762379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1367/2692] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:45.765231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec2602-bbda-440f-a5a3-a95e4b8f3ee0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:45.768836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-40ec2602-bbda-440f-a5a3-a95e4b8f3ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:45.808329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec2602-bbda-440f-a5a3-a95e4b8f3ee0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:45.831027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1326/2693] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:45.831027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457b7e9-556f-4918-9b4c-8d8cd5f59d4f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:45.831027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6457b7e9-556f-4918-9b4c-8d8cd5f59d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:45.848896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457b7e9-556f-4918-9b4c-8d8cd5f59d4f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:45.855152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b8f7d-78a8-49a2-b5b3-cdc3c4e5122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:45.856677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b8f7d-78a8-49a2-b5b3-cdc3c4e5122d admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:45.856677 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b8f7d-78a8-49a2-b5b3-cdc3c4e5122d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:45.856927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b8f7d-78a8-49a2-b5b3-cdc3c4e5122d admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:45.857459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b8f7d-78a8-49a2-b5b3-cdc3c4e5122d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:45.859807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b8f7d-78a8-49a2-b5b3-cdc3c4e5122d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:45.863757 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:45.903294 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1327/2694] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:45 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:45.904598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b8f7d-78a8-49a2-b5b3-cdc3c4e5122d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:45.907549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-843b8f7d-78a8-49a2-b5b3-cdc3c4e5122d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:45.928180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:45.929263 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:45.930656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:45.931168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] PATH_INFO: `/v3/projects/f97acc4269294ff4be07755871d3bb7b` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:45.933058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97acc4269294ff4be07755871d3bb7b)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:45.936075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:46.004108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b8f7d-78a8-49a2-b5b3-cdc3c4e5122d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:46.020390 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97acc4269294ff4be07755871d3bb7b', 'request_id': 'req-f57d37da-8267-4583-b144-8d1297ae5573'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:46.033694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97acc4269294ff4be07755871d3bb7b', 'request_id': 'req-f57d37da-8267-4583-b144-8d1297ae5573'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:46.047492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97acc4269294ff4be07755871d3bb7b', 'request_id': 'req-f57d37da-8267-4583-b144-8d1297ae5573'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:46.067154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] The token cache is being invalidate because project f97acc4269294ff4be07755871d3bb7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:46.067580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:46.068321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:46.070349 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1368/2695] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:45 2022] DELETE /identity/v3/projects/f97acc4269294ff4be07755871d3bb7b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:46.071602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:46.072578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145d13a-d167-4d6c-8ce5-73adadcb57c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:46.073697 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:46.073996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145d13a-d167-4d6c-8ce5-73adadcb57c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:46.074341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145d13a-d167-4d6c-8ce5-73adadcb57c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:46.074678 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145d13a-d167-4d6c-8ce5-73adadcb57c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:46.078256 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7145d13a-d167-4d6c-8ce5-73adadcb57c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:46.191544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d37da-8267-4583-b144-8d1297ae5573 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:46.197920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361c066-f1ac-4ea7-8de1-860a941a65c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:46.198646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361c066-f1ac-4ea7-8de1-860a941a65c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:46.198939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361c066-f1ac-4ea7-8de1-860a941a65c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:46.199226 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361c066-f1ac-4ea7-8de1-860a941a65c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:46.199974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361c066-f1ac-4ea7-8de1-860a941a65c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:46.202081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361c066-f1ac-4ea7-8de1-860a941a65c5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:46.210494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145d13a-d167-4d6c-8ce5-73adadcb57c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:46.213167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145d13a-d167-4d6c-8ce5-73adadcb57c0 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:46.226972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1369/2696] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:46 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:46.239614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1328/2697] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:46.240521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361c066-f1ac-4ea7-8de1-860a941a65c5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:46.255689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361c066-f1ac-4ea7-8de1-860a941a65c5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:46.265491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5457b6-0ba5-4cf3-9465-c0f2d7fbab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:46.267612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5457b6-0ba5-4cf3-9465-c0f2d7fbab8a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:46.268071 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5457b6-0ba5-4cf3-9465-c0f2d7fbab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:46.268491 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5457b6-0ba5-4cf3-9465-c0f2d7fbab8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:46.269954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5457b6-0ba5-4cf3-9465-c0f2d7fbab8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:46.272699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5457b6-0ba5-4cf3-9465-c0f2d7fbab8a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:46.286620 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:46.298524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1370/2698] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:46.301766 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145d13a-d167-4d6c-8ce5-73adadcb57c0 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:46.320857 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145d13a-d167-4d6c-8ce5-73adadcb57c0 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:46.328135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e22b-fb9b-4e38-b7bf-b8c942c674a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:46.329325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e22b-fb9b-4e38-b7bf-b8c942c674a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:46.329761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e22b-fb9b-4e38-b7bf-b8c942c674a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:46.330187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e22b-fb9b-4e38-b7bf-b8c942c674a8 admin admin] PATH_INFO: `/v3/projects/95df364cf8a44185864291e4a856f3d2/users/9bddb1e3ad7a4ad98f0c25ca4c7b9a1a/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:46.354568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e22b-fb9b-4e38-b7bf-b8c942c674a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95df364cf8a44185864291e4a856f3d2, user_id=9bddb1e3ad7a4ad98f0c25ca4c7b9a1a, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:46.358463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e22b-fb9b-4e38-b7bf-b8c942c674a8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:46.378254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1329/2699] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:46 2022] PUT /identity/v3/projects/95df364cf8a44185864291e4a856f3d2/users/9bddb1e3ad7a4ad98f0c25ca4c7b9a1a/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:46.382972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5457b6-0ba5-4cf3-9465-c0f2d7fbab8a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:46.394160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5457b6-0ba5-4cf3-9465-c0f2d7fbab8a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:46.509946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347bd85-2ab7-4f5d-a831-210ab830abbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:46.510786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347bd85-2ab7-4f5d-a831-210ab830abbf admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:46.511190 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347bd85-2ab7-4f5d-a831-210ab830abbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:46.511566 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347bd85-2ab7-4f5d-a831-210ab830abbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:46.512763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347bd85-2ab7-4f5d-a831-210ab830abbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:46.515123 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347bd85-2ab7-4f5d-a831-210ab830abbf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:46.528352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1371/2700] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:46.533502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e22b-fb9b-4e38-b7bf-b8c942c674a8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:46.555645 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e22b-fb9b-4e38-b7bf-b8c942c674a8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:46.564284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71a13-5658-4637-a5cd-e955d503286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:46.565329 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71a13-5658-4637-a5cd-e955d503286f admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:46.565685 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71a13-5658-4637-a5cd-e955d503286f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:46.566068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71a13-5658-4637-a5cd-e955d503286f admin admin] PATH_INFO: `/v3/projects/95df364cf8a44185864291e4a856f3d2/users/9bddb1e3ad7a4ad98f0c25ca4c7b9a1a/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:46.568356 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f71a13-5658-4637-a5cd-e955d503286f admin admin] RBAC: Authorizing `identity:create_grant(project_id=95df364cf8a44185864291e4a856f3d2, user_id=9bddb1e3ad7a4ad98f0c25ca4c7b9a1a, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:46.571287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f71a13-5658-4637-a5cd-e955d503286f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:46.584307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1330/2701] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:46 2022] PUT /identity/v3/projects/95df364cf8a44185864291e4a856f3d2/users/9bddb1e3ad7a4ad98f0c25ca4c7b9a1a/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:47.697518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d67de5-edee-472c-82da-905b6bf55d0c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:47.698400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d67de5-edee-472c-82da-905b6bf55d0c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:47.698400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d67de5-edee-472c-82da-905b6bf55d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:47.754363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d7d67de5-edee-472c-82da-905b6bf55d0c None None] MFA Rules not processed for user `175c6e4093f34acf97165852c125ab6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:47.756568 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d7d67de5-edee-472c-82da-905b6bf55d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:47.879141 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1372/2702] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:47.884454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71a13-5658-4637-a5cd-e955d503286f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:47.889175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b6f71a13-5658-4637-a5cd-e955d503286f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:47.907002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:47.992897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71a13-5658-4637-a5cd-e955d503286f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:47.998268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007b91c-d975-495c-8211-b1c536db8733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:47.999610 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007b91c-d975-495c-8211-b1c536db8733 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:47.999911 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007b91c-d975-495c-8211-b1c536db8733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:48.000181 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007b91c-d975-495c-8211-b1c536db8733 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:48.000896 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007b91c-d975-495c-8211-b1c536db8733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:48.002929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007b91c-d975-495c-8211-b1c536db8733 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:48.044153 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1331/2703] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:47 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:48.046334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d67de5-edee-472c-82da-905b6bf55d0c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:48.064252 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d67de5-edee-472c-82da-905b6bf55d0c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:48.170480 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b40cd2-d95b-4534-a7e0-191e261948b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:48.171227 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b40cd2-d95b-4534-a7e0-191e261948b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:48.171593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b40cd2-d95b-4534-a7e0-191e261948b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:48.172084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b40cd2-d95b-4534-a7e0-191e261948b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:48.172818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b40cd2-d95b-4534-a7e0-191e261948b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:48.176366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b40cd2-d95b-4534-a7e0-191e261948b4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:48.185407 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2f9a3-f516-4e8b-acf5-fd14c434b916 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:48.185690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2f9a3-f516-4e8b-acf5-fd14c434b916 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:48.186074 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2f9a3-f516-4e8b-acf5-fd14c434b916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:48.209038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1373/2704] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:48 2022] POST /identity/v3/users => generated 309 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:48.210318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b40cd2-d95b-4534-a7e0-191e261948b4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:48.222176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9ea2f9a3-f516-4e8b-acf5-fd14c434b916 None None] MFA Rules not processed for user `9bddb1e3ad7a4ad98f0c25ca4c7b9a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:48.223284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b40cd2-d95b-4534-a7e0-191e261948b4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:48.223494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9ea2f9a3-f516-4e8b-acf5-fd14c434b916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:48.230792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c214-68d6-4a3a-8ee1-cd69361d2836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:48.231413 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c214-68d6-4a3a-8ee1-cd69361d2836 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:48.231691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c214-68d6-4a3a-8ee1-cd69361d2836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:48.232171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c214-68d6-4a3a-8ee1-cd69361d2836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:48.232938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56c214-68d6-4a3a-8ee1-cd69361d2836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:48.234850 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56c214-68d6-4a3a-8ee1-cd69361d2836 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:48.249763 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1374/2705] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:48.255523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c214-68d6-4a3a-8ee1-cd69361d2836 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:48.266243 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:48.274438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c214-68d6-4a3a-8ee1-cd69361d2836 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:48.286471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82027f58-6e18-4598-8055-405f8b3a29bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:48.287562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82027f58-6e18-4598-8055-405f8b3a29bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:48.287975 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82027f58-6e18-4598-8055-405f8b3a29bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:48.288295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82027f58-6e18-4598-8055-405f8b3a29bf admin admin] PATH_INFO: `/v3/projects/55a6065189b7482d8da9f62b66a286e7/users/07d571e2a19043ed8fc6837c36ba288c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:48.312261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82027f58-6e18-4598-8055-405f8b3a29bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a6065189b7482d8da9f62b66a286e7, user_id=07d571e2a19043ed8fc6837c36ba288c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:48.318140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82027f58-6e18-4598-8055-405f8b3a29bf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:48.327301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1332/2706] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:48 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:48.342763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2f9a3-f516-4e8b-acf5-fd14c434b916 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:48.344485 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9ea2f9a3-f516-4e8b-acf5-fd14c434b916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:48.351342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1375/2707] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:48 2022] PUT /identity/v3/projects/55a6065189b7482d8da9f62b66a286e7/users/07d571e2a19043ed8fc6837c36ba288c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:48.356626 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82027f58-6e18-4598-8055-405f8b3a29bf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:48.446463 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2f9a3-f516-4e8b-acf5-fd14c434b916 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:48.474116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82027f58-6e18-4598-8055-405f8b3a29bf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:48.481000 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75f0f6-de5a-463c-856b-c7b91a35cd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:48.481689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75f0f6-de5a-463c-856b-c7b91a35cd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:48.482006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75f0f6-de5a-463c-856b-c7b91a35cd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:48.482282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75f0f6-de5a-463c-856b-c7b91a35cd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:48.483047 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75f0f6-de5a-463c-856b-c7b91a35cd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:48.484716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75f0f6-de5a-463c-856b-c7b91a35cd92 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:48.508430 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1376/2708] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:48.514671 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75f0f6-de5a-463c-856b-c7b91a35cd92 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:48.520067 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:48.532241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75f0f6-de5a-463c-856b-c7b91a35cd92 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:48.535509 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:48.536284 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:48.536736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:48.537132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:48.538137 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c59a9-b806-4b75-914b-b9df9c510b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:48.539707 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c59a9-b806-4b75-914b-b9df9c510b22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:48.539991 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c59a9-b806-4b75-914b-b9df9c510b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:48.540302 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c59a9-b806-4b75-914b-b9df9c510b22 admin admin] PATH_INFO: `/v3/projects/55a6065189b7482d8da9f62b66a286e7/users/07d571e2a19043ed8fc6837c36ba288c/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:48.541642 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:48.542492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c59a9-b806-4b75-914b-b9df9c510b22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a6065189b7482d8da9f62b66a286e7, user_id=07d571e2a19043ed8fc6837c36ba288c, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:48.545230 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c59a9-b806-4b75-914b-b9df9c510b22 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:48.566135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1377/2709] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:48 2022] PUT /identity/v3/projects/55a6065189b7482d8da9f62b66a286e7/users/07d571e2a19043ed8fc6837c36ba288c/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:48.657121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:48.660108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:48.680773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1333/2710] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:30:49.662068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309fb04-e0e0-4503-866d-ea94d8573122 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:49.662068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309fb04-e0e0-4503-866d-ea94d8573122 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:49.662625 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309fb04-e0e0-4503-866d-ea94d8573122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:49.720828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9309fb04-e0e0-4503-866d-ea94d8573122 None None] MFA Rules not processed for user `07d571e2a19043ed8fc6837c36ba288c`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:49.723606 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9309fb04-e0e0-4503-866d-ea94d8573122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:49.829602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1378/2711] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:49 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:50.355995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:50.357519 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:50.370211 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4c4a-fd5a-4bbe-80cd-bbcee51357b3 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:50.436268 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8efdd-c905-409d-8ec3-a6d69a8503c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:50.436938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8efdd-c905-409d-8ec3-a6d69a8503c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:50.437325 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8efdd-c905-409d-8ec3-a6d69a8503c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:50.437675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8efdd-c905-409d-8ec3-a6d69a8503c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:50.524456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8efdd-c905-409d-8ec3-a6d69a8503c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:50.526138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8efdd-c905-409d-8ec3-a6d69a8503c8 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:50.537428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1334/2712] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:51.191561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309fb04-e0e0-4503-866d-ea94d8573122 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:51.274084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:51.293169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309fb04-e0e0-4503-866d-ea94d8573122 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:51.298103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:51.299037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:51.299320 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:51.299604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] PATH_INFO: `/v3/users/175c6e4093f34acf97165852c125ab6e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:51.300920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=175c6e4093f34acf97165852c125ab6e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:51.302955 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:51.365720 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175c6e4093f34acf97165852c125ab6e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.374358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175c6e4093f34acf97165852c125ab6e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.375105 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175c6e4093f34acf97165852c125ab6e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.384322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175c6e4093f34acf97165852c125ab6e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.399549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175c6e4093f34acf97165852c125ab6e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.407483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1379/2713] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:51 2022] DELETE /identity/v3/users/175c6e4093f34acf97165852c125ab6e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:51.616193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8efdd-c905-409d-8ec3-a6d69a8503c8 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:51.635124 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8efdd-c905-409d-8ec3-a6d69a8503c8 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:51.709793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:51.730763 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:51.731876 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:51.732220 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:51.732720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] PATH_INFO: `/v3/projects/f1093cc1cad045eca64cb2047cf227f4` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:51.734387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1093cc1cad045eca64cb2047cf227f4)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:51.736612 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:51.794762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1093cc1cad045eca64cb2047cf227f4', 'request_id': 'req-a493a087-ea41-4b51-a647-0db6e981a246'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.806527 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1093cc1cad045eca64cb2047cf227f4', 'request_id': 'req-a493a087-ea41-4b51-a647-0db6e981a246'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.816826 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1093cc1cad045eca64cb2047cf227f4', 'request_id': 'req-a493a087-ea41-4b51-a647-0db6e981a246'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.839458 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] The token cache is being invalidate because project f1093cc1cad045eca64cb2047cf227f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:51.839865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.840941 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:51.843348 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1335/2714] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:51 2022] DELETE /identity/v3/projects/f1093cc1cad045eca64cb2047cf227f4 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:51.848835 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:51.849611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:51.868177 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee49823-a94c-4702-b665-fc9674a71492 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:51.954605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:51.955613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:51.955963 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:51.956306 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] PATH_INFO: `/v3/users/07d571e2a19043ed8fc6837c36ba288c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:51.957823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d571e2a19043ed8fc6837c36ba288c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:51.960088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:52.026186 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d571e2a19043ed8fc6837c36ba288c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.041456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d571e2a19043ed8fc6837c36ba288c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.042284 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d571e2a19043ed8fc6837c36ba288c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.052308 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d571e2a19043ed8fc6837c36ba288c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.063501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:52.072492 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d571e2a19043ed8fc6837c36ba288c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.086031 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1380/2715] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:51 2022] DELETE /identity/v3/users/07d571e2a19043ed8fc6837c36ba288c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:52.124897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:52.126667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:52.151979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493a087-ea41-4b51-a647-0db6e981a246 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:52.301127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:52.302160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:52.302643 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:52.303026 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] PATH_INFO: `/v3/users/b572ed434c714fa78885deb417df864c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:52.306127 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b572ed434c714fa78885deb417df864c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:52.308272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:52.409901 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b572ed434c714fa78885deb417df864c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.417246 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:52.419730 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:52.427958 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b572ed434c714fa78885deb417df864c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.428904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b572ed434c714fa78885deb417df864c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.444264 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b572ed434c714fa78885deb417df864c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.445257 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2204cd7c-6f35-458b-b3f8-9240fc71c228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b572ed434c714fa78885deb417df864c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.446292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2b8ce-7208-4b89-98f1-c27085af7306 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:52.450193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:52.457757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:52.459181 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:52.459584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:52.460110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] PATH_INFO: `/v3/projects/55a6065189b7482d8da9f62b66a286e7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:52.462624 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] RBAC: Authorizing `identity:delete_project(project_id=55a6065189b7482d8da9f62b66a286e7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:52.465021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:52.493285 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1336/2716] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:52 2022] DELETE /identity/v3/users/b572ed434c714fa78885deb417df864c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:30:52.500369 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783b308-bb6d-46b8-ab0a-bbaf1f8bab7d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:52.500856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783b308-bb6d-46b8-ab0a-bbaf1f8bab7d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:52.501368 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783b308-bb6d-46b8-ab0a-bbaf1f8bab7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:52.535624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2783b308-bb6d-46b8-ab0a-bbaf1f8bab7d None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:52.537637 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2783b308-bb6d-46b8-ab0a-bbaf1f8bab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:52.556612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55a6065189b7482d8da9f62b66a286e7', 'request_id': 'req-71c6948f-5f85-4e51-b547-27eae5fc508f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.571350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55a6065189b7482d8da9f62b66a286e7', 'request_id': 'req-71c6948f-5f85-4e51-b547-27eae5fc508f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.586028 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55a6065189b7482d8da9f62b66a286e7', 'request_id': 'req-71c6948f-5f85-4e51-b547-27eae5fc508f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.622393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] The token cache is being invalidate because project 55a6065189b7482d8da9f62b66a286e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:52.622393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.623504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71c6948f-5f85-4e51-b547-27eae5fc508f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:52.626352 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1381/2717] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:52 2022] DELETE /identity/v3/projects/55a6065189b7482d8da9f62b66a286e7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:52.633497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d76a7-a30d-4b7a-8b8f-b2ac9c4d6b6a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:52.633941 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d76a7-a30d-4b7a-8b8f-b2ac9c4d6b6a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:52.634121 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d76a7-a30d-4b7a-8b8f-b2ac9c4d6b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:52.662970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1337/2718] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:52.665135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6c8d76a7-a30d-4b7a-8b8f-b2ac9c4d6b6a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:52.666961 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6c8d76a7-a30d-4b7a-8b8f-b2ac9c4d6b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:52.679388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783b308-bb6d-46b8-ab0a-bbaf1f8bab7d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:52.682082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2783b308-bb6d-46b8-ab0a-bbaf1f8bab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:52.682781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1382/2719] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:30:52.691455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b35053-0036-44dd-aaa7-92c1d51c1711 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:52.691806 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b35053-0036-44dd-aaa7-92c1d51c1711 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:52.692093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b35053-0036-44dd-aaa7-92c1d51c1711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:52.717359 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-24b35053-0036-44dd-aaa7-92c1d51c1711 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:52.718971 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-24b35053-0036-44dd-aaa7-92c1d51c1711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:52.731710 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1383/2720] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:52.737305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b35053-0036-44dd-aaa7-92c1d51c1711 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:52.753384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b35053-0036-44dd-aaa7-92c1d51c1711 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:52.761798 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed8b3f-bc34-47ce-8ab9-a1550c667a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:52.762954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed8b3f-bc34-47ce-8ab9-a1550c667a5b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:52.763686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed8b3f-bc34-47ce-8ab9-a1550c667a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:52.764173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed8b3f-bc34-47ce-8ab9-a1550c667a5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:52.765576 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed8b3f-bc34-47ce-8ab9-a1550c667a5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:30:52.766478 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed8b3f-bc34-47ce-8ab9-a1550c667a5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:52.769633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed8b3f-bc34-47ce-8ab9-a1550c667a5b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:52.792162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1384/2721] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:30:52.797480 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed8b3f-bc34-47ce-8ab9-a1550c667a5b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:52.803402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783b308-bb6d-46b8-ab0a-bbaf1f8bab7d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:52.808929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d6545-c00c-4dd1-b320-100f60230d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:52.809523 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6545-c00c-4dd1-b320-100f60230d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:52.809845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6545-c00c-4dd1-b320-100f60230d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:52.810132 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6545-c00c-4dd1-b320-100f60230d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:52.811151 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed8b3f-bc34-47ce-8ab9-a1550c667a5b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:52.816728 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277a2aa-68ed-4e52-8cfe-987ef376fab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:52.817647 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277a2aa-68ed-4e52-8cfe-987ef376fab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:52.818118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277a2aa-68ed-4e52-8cfe-987ef376fab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:52.818593 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277a2aa-68ed-4e52-8cfe-987ef376fab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:52.819396 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277a2aa-68ed-4e52-8cfe-987ef376fab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:52.821658 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d6545-c00c-4dd1-b320-100f60230d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:52.822622 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277a2aa-68ed-4e52-8cfe-987ef376fab2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:52.823159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d6545-c00c-4dd1-b320-100f60230d17 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:52.834697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1338/2722] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:52.864204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1385/2723] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:52 2022] POST /identity/v3/projects => generated 391 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:52.870304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d6545-c00c-4dd1-b320-100f60230d17 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:52.887420 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d6545-c00c-4dd1-b320-100f60230d17 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:52.998812 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15225d-1212-468e-ab2e-79590a2ff45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:52.999993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15225d-1212-468e-ab2e-79590a2ff45a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:53.000497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15225d-1212-468e-ab2e-79590a2ff45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:53.000999 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15225d-1212-468e-ab2e-79590a2ff45a admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:53.002389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15225d-1212-468e-ab2e-79590a2ff45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:53.009108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15225d-1212-468e-ab2e-79590a2ff45a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:53.052111 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1339/2724] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:52 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:53.053198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277a2aa-68ed-4e52-8cfe-987ef376fab2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:53.066079 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277a2aa-68ed-4e52-8cfe-987ef376fab2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:53.071814 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78ea99-22af-4582-9dce-39cac8ac7f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:53.072569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78ea99-22af-4582-9dce-39cac8ac7f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:53.072916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78ea99-22af-4582-9dce-39cac8ac7f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:53.073435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78ea99-22af-4582-9dce-39cac8ac7f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:53.074518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78ea99-22af-4582-9dce-39cac8ac7f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:53.076297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78ea99-22af-4582-9dce-39cac8ac7f9d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:53.093678 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1386/2725] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:53.095105 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15225d-1212-468e-ab2e-79590a2ff45a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:53.105997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15225d-1212-468e-ab2e-79590a2ff45a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:53.111243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e895c-5d99-451f-b7b8-dd610b00e19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:53.111928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e895c-5d99-451f-b7b8-dd610b00e19b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:53.112379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e895c-5d99-451f-b7b8-dd610b00e19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:53.112515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e895c-5d99-451f-b7b8-dd610b00e19b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:53.113207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e895c-5d99-451f-b7b8-dd610b00e19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:53.114881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e895c-5d99-451f-b7b8-dd610b00e19b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:53.129046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1340/2726] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:53.134854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78ea99-22af-4582-9dce-39cac8ac7f9d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:53.146417 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78ea99-22af-4582-9dce-39cac8ac7f9d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:53.152956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536ba99-0b20-43fb-9659-e496b80aca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:53.154304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536ba99-0b20-43fb-9659-e496b80aca79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:53.154721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536ba99-0b20-43fb-9659-e496b80aca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:53.155185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536ba99-0b20-43fb-9659-e496b80aca79 admin admin] PATH_INFO: `/v3/projects/434669af7ccb48e7b84f3905074d47ab/users/22c8f18323764437b4e449e974024f8b/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:53.175335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536ba99-0b20-43fb-9659-e496b80aca79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=434669af7ccb48e7b84f3905074d47ab, user_id=22c8f18323764437b4e449e974024f8b, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:53.178168 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536ba99-0b20-43fb-9659-e496b80aca79 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:53.195853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1387/2727] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:53 2022] PUT /identity/v3/projects/434669af7ccb48e7b84f3905074d47ab/users/22c8f18323764437b4e449e974024f8b/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:53.208418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf2a3-5fe3-4886-a96a-38bc5b755eff None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:53.208924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf2a3-5fe3-4886-a96a-38bc5b755eff None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:53.209431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf2a3-5fe3-4886-a96a-38bc5b755eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:53.283201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-80dbf2a3-5fe3-4886-a96a-38bc5b755eff None None] MFA Rules not processed for user `22c8f18323764437b4e449e974024f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:53.285454 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-80dbf2a3-5fe3-4886-a96a-38bc5b755eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:53.288674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536ba99-0b20-43fb-9659-e496b80aca79 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:53.293607 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3536ba99-0b20-43fb-9659-e496b80aca79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:53.296573 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:53.408261 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1341/2728] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:53.408482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:53.410235 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf2a3-5fe3-4886-a96a-38bc5b755eff None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:53.436360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536ba99-0b20-43fb-9659-e496b80aca79 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:53.443486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:53.444984 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:53.445639 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:53.446111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] PATH_INFO: `/v3/projects/0698f89e78e84a19847c1b05bc05311b` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:53.448185 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0698f89e78e84a19847c1b05bc05311b)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:53.451828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:53.547846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0698f89e78e84a19847c1b05bc05311b', 'request_id': 'req-c68a14eb-bec0-4082-a08e-2795b28f916f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:53.563948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf2a3-5fe3-4886-a96a-38bc5b755eff None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:53.568223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0698f89e78e84a19847c1b05bc05311b', 'request_id': 'req-c68a14eb-bec0-4082-a08e-2795b28f916f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:53.587845 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0698f89e78e84a19847c1b05bc05311b', 'request_id': 'req-c68a14eb-bec0-4082-a08e-2795b28f916f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:53.616553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] The token cache is being invalidate because project 0698f89e78e84a19847c1b05bc05311b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:53.622839 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:53.626732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:53.631149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1388/2729] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:53 2022] DELETE /identity/v3/projects/0698f89e78e84a19847c1b05bc05311b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:53.737347 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190b6be-2fec-482a-a0a9-b4510b78e86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:53.739187 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190b6be-2fec-482a-a0a9-b4510b78e86a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:53.739934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190b6be-2fec-482a-a0a9-b4510b78e86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:53.740438 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190b6be-2fec-482a-a0a9-b4510b78e86a admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:53.743506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190b6be-2fec-482a-a0a9-b4510b78e86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:53.749913 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190b6be-2fec-482a-a0a9-b4510b78e86a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:53.806015 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1342/2730] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:30:53 2022] POST /identity/v3/projects => generated 391 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:30:53.806139 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:53.807794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:53.823262 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190b6be-2fec-482a-a0a9-b4510b78e86a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:53.825178 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5190b6be-2fec-482a-a0a9-b4510b78e86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:53.841580 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a14eb-bec0-4082-a08e-2795b28f916f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:53.917367 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:53.958155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190b6be-2fec-482a-a0a9-b4510b78e86a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:53.965572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335c4a-8aeb-4df3-9454-6e064596fc3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:53.967074 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335c4a-8aeb-4df3-9454-6e064596fc3e service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:53.967778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335c4a-8aeb-4df3-9454-6e064596fc3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:53.968247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335c4a-8aeb-4df3-9454-6e064596fc3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:53.971179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0bef-65ef-468e-8292-9a200362b27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:53.971344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e9335c4a-8aeb-4df3-9454-6e064596fc3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:53.972253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0bef-65ef-468e-8292-9a200362b27e admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:53.972706 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0bef-65ef-468e-8292-9a200362b27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:53.973360 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0bef-65ef-468e-8292-9a200362b27e admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:53.974700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0bef-65ef-468e-8292-9a200362b27e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:53.978486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0bef-65ef-468e-8292-9a200362b27e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:54.008279 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1389/2731] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:30:53 2022] POST /identity/v3/users => generated 302 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:30:54.012578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0bef-65ef-468e-8292-9a200362b27e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:54.028958 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0bef-65ef-468e-8292-9a200362b27e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:54.034023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cb968-7978-42e8-9351-b4682e8fe84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:54.034628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cb968-7978-42e8-9351-b4682e8fe84b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:54.034942 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cb968-7978-42e8-9351-b4682e8fe84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:54.035216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cb968-7978-42e8-9351-b4682e8fe84b admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:54.036089 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cb968-7978-42e8-9351-b4682e8fe84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:54.037585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cb968-7978-42e8-9351-b4682e8fe84b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:54.050247 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1390/2732] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:30:54.055520 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cb968-7978-42e8-9351-b4682e8fe84b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:54.073083 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cb968-7978-42e8-9351-b4682e8fe84b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:54.077944 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9335c4a-8aeb-4df3-9454-6e064596fc3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:54.079518 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9335c4a-8aeb-4df3-9454-6e064596fc3e service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:54.082483 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec4f34-5075-4028-a98f-cf0f5b64f518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:54.084020 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec4f34-5075-4028-a98f-cf0f5b64f518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:54.084533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec4f34-5075-4028-a98f-cf0f5b64f518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:54.085130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec4f34-5075-4028-a98f-cf0f5b64f518 admin admin] PATH_INFO: `/v3/projects/2094bd9c890143439983b920897e009c/users/3703cfe895054aad8c27a19ad41cdfaf/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:54.092651 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1343/2733] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:54.113327 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec4f34-5075-4028-a98f-cf0f5b64f518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2094bd9c890143439983b920897e009c, user_id=3703cfe895054aad8c27a19ad41cdfaf, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:54.120894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec4f34-5075-4028-a98f-cf0f5b64f518 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:54.150245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1391/2734] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:54 2022] PUT /identity/v3/projects/2094bd9c890143439983b920897e009c/users/3703cfe895054aad8c27a19ad41cdfaf/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:54.156045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335c4a-8aeb-4df3-9454-6e064596fc3e service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:54.167948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335c4a-8aeb-4df3-9454-6e064596fc3e service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:54.241349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad355b-bfd8-4a67-bf80-30ba3947248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:54.242809 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad355b-bfd8-4a67-bf80-30ba3947248b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:54.243279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad355b-bfd8-4a67-bf80-30ba3947248b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:54.243751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad355b-bfd8-4a67-bf80-30ba3947248b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:54.244517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad355b-bfd8-4a67-bf80-30ba3947248b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:54.246242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad355b-bfd8-4a67-bf80-30ba3947248b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:54.263754 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1344/2735] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:30:54.265330 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec4f34-5075-4028-a98f-cf0f5b64f518 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:54.279144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec4f34-5075-4028-a98f-cf0f5b64f518 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:54.285228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de65b4-2329-4518-bd87-e6a8c0fab2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:54.286207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de65b4-2329-4518-bd87-e6a8c0fab2ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:54.286531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de65b4-2329-4518-bd87-e6a8c0fab2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:54.286800 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de65b4-2329-4518-bd87-e6a8c0fab2ea admin admin] PATH_INFO: `/v3/projects/2094bd9c890143439983b920897e009c/users/3703cfe895054aad8c27a19ad41cdfaf/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:54.290218 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de65b4-2329-4518-bd87-e6a8c0fab2ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2094bd9c890143439983b920897e009c, user_id=3703cfe895054aad8c27a19ad41cdfaf, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:54.292721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de65b4-2329-4518-bd87-e6a8c0fab2ea admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:54.305683 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1392/2736] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:30:54 2022] PUT /identity/v3/projects/2094bd9c890143439983b920897e009c/users/3703cfe895054aad8c27a19ad41cdfaf/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:54.317696 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33f86-1c06-4a85-ab46-0c413717c00a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:54.318027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33f86-1c06-4a85-ab46-0c413717c00a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:54.318313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33f86-1c06-4a85-ab46-0c413717c00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:54.353884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-73d33f86-1c06-4a85-ab46-0c413717c00a None None] MFA Rules not processed for user `3703cfe895054aad8c27a19ad41cdfaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:54.355250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-73d33f86-1c06-4a85-ab46-0c413717c00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:54.467386 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1345/2737] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:54.483522 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de65b4-2329-4518-bd87-e6a8c0fab2ea admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:54.486418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-26de65b4-2329-4518-bd87-e6a8c0fab2ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:54.519023 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:54.567372 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de65b4-2329-4518-bd87-e6a8c0fab2ea admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:54.574676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8fcb4-9978-4bd7-a25c-9ae2267643d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:54.575828 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8fcb4-9978-4bd7-a25c-9ae2267643d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:54.576307 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8fcb4-9978-4bd7-a25c-9ae2267643d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:54.576802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8fcb4-9978-4bd7-a25c-9ae2267643d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:54.595699 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8fcb4-9978-4bd7-a25c-9ae2267643d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:54.598605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8fcb4-9978-4bd7-a25c-9ae2267643d5 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:54.618700 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1393/2738] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:54.705842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33f86-1c06-4a85-ab46-0c413717c00a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:54.708190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-73d33f86-1c06-4a85-ab46-0c413717c00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:54.731630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33f86-1c06-4a85-ab46-0c413717c00a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:54.737497 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a210e-f661-4b5b-a066-6d9ef8a21910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:54.738558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a210e-f661-4b5b-a066-6d9ef8a21910 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:54.738983 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a210e-f661-4b5b-a066-6d9ef8a21910 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:54.739381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a210e-f661-4b5b-a066-6d9ef8a21910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:54.743058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-785a210e-f661-4b5b-a066-6d9ef8a21910 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:54.753331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:54.847209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a210e-f661-4b5b-a066-6d9ef8a21910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:54.849688 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a210e-f661-4b5b-a066-6d9ef8a21910 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:54.868709 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1346/2739] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:59.014135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8fcb4-9978-4bd7-a25c-9ae2267643d5 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:59.088552 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8fcb4-9978-4bd7-a25c-9ae2267643d5 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:59.094585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:59.096317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:59.096785 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:59.097278 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] PATH_INFO: `/v3/users/22c8f18323764437b4e449e974024f8b` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:59.099633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22c8f18323764437b4e449e974024f8b)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:59.103457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:59.175063 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22c8f18323764437b4e449e974024f8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.183356 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22c8f18323764437b4e449e974024f8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.184236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22c8f18323764437b4e449e974024f8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.193067 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22c8f18323764437b4e449e974024f8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.208050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22c8f18323764437b4e449e974024f8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.215488 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1394/2740] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:59 2022] DELETE /identity/v3/users/22c8f18323764437b4e449e974024f8b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:59.221399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21245aec-c7db-41cd-8a8f-d34c86a06178 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:59.222135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21245aec-c7db-41cd-8a8f-d34c86a06178 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:59.222416 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21245aec-c7db-41cd-8a8f-d34c86a06178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:59.252794 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-21245aec-c7db-41cd-8a8f-d34c86a06178 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:30:59.254937 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-21245aec-c7db-41cd-8a8f-d34c86a06178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:59.336209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1347/2741] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:30:59.353290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:59.354930 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:59.408924 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:30:59.428339 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8f099-bf15-4c34-bfdc-93705f51bae1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:59.435130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5a77d-d25a-4e97-937c-1e5c645ea9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:59.435805 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5a77d-d25a-4e97-937c-1e5c645ea9df service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:59.436289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5a77d-d25a-4e97-937c-1e5c645ea9df service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:59.436582 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5a77d-d25a-4e97-937c-1e5c645ea9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:59.449336 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5a77d-d25a-4e97-937c-1e5c645ea9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:59.450873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5a77d-d25a-4e97-937c-1e5c645ea9df service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:59.465329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1395/2742] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:30:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:30:59.848059 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21245aec-c7db-41cd-8a8f-d34c86a06178 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:59.849446 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-21245aec-c7db-41cd-8a8f-d34c86a06178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:30:59.864405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21245aec-c7db-41cd-8a8f-d34c86a06178 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:30:59.871107 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:30:59.871995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:30:59.872352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:30:59.872558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] PATH_INFO: `/v3/projects/434669af7ccb48e7b84f3905074d47ab` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:30:59.873866 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] RBAC: Authorizing `identity:delete_project(project_id=434669af7ccb48e7b84f3905074d47ab)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:30:59.875865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:30:59.946614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '434669af7ccb48e7b84f3905074d47ab', 'request_id': 'req-0eea84b4-3e86-47d7-a1ba-53879cbb224f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.956307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '434669af7ccb48e7b84f3905074d47ab', 'request_id': 'req-0eea84b4-3e86-47d7-a1ba-53879cbb224f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.964928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '434669af7ccb48e7b84f3905074d47ab', 'request_id': 'req-0eea84b4-3e86-47d7-a1ba-53879cbb224f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.979324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] The token cache is being invalidate because project 434669af7ccb48e7b84f3905074d47ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:30:59.979324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.979903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:30:59.982121 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1348/2743] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:30:59 2022] DELETE /identity/v3/projects/434669af7ccb48e7b84f3905074d47ab => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:30:59.986821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5a77d-d25a-4e97-937c-1e5c645ea9df service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:30:59.998689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5a77d-d25a-4e97-937c-1e5c645ea9df service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:00.070314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:00.071366 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:00.071650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:00.071939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] PATH_INFO: `/v3/users/3703cfe895054aad8c27a19ad41cdfaf` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:00.073408 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3703cfe895054aad8c27a19ad41cdfaf)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:00.075466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:00.137525 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3703cfe895054aad8c27a19ad41cdfaf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.145954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3703cfe895054aad8c27a19ad41cdfaf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.146711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3703cfe895054aad8c27a19ad41cdfaf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.157147 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3703cfe895054aad8c27a19ad41cdfaf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.173659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5b2b87a9-a39f-40c1-a6df-a9b6c61aa4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3703cfe895054aad8c27a19ad41cdfaf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.181248 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1396/2744] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:30:59 2022] DELETE /identity/v3/users/3703cfe895054aad8c27a19ad41cdfaf => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:00.555504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:00.557797 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:00.584069 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea84b4-3e86-47d7-a1ba-53879cbb224f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:00.625814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:00.666048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:00.667381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:00.667855 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:00.668335 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] PATH_INFO: `/v3/projects/2094bd9c890143439983b920897e009c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:00.670742 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2094bd9c890143439983b920897e009c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:00.673816 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:00.739190 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2094bd9c890143439983b920897e009c', 'request_id': 'req-782b16ff-8093-4f14-a047-fbd928224e3a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.749081 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2094bd9c890143439983b920897e009c', 'request_id': 'req-782b16ff-8093-4f14-a047-fbd928224e3a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.756659 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2094bd9c890143439983b920897e009c', 'request_id': 'req-782b16ff-8093-4f14-a047-fbd928224e3a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.770217 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] The token cache is being invalidate because project 2094bd9c890143439983b920897e009c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:00.770490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.771341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-782b16ff-8093-4f14-a047-fbd928224e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:00.774078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1349/2745] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:00 2022] DELETE /identity/v3/projects/2094bd9c890143439983b920897e009c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:00.779860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d931ad-cc7f-4721-9d3b-5f6af8229394 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:00.780189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d931ad-cc7f-4721-9d3b-5f6af8229394 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:00.780863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d931ad-cc7f-4721-9d3b-5f6af8229394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:00.802834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-19d931ad-cc7f-4721-9d3b-5f6af8229394 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:00.804006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-19d931ad-cc7f-4721-9d3b-5f6af8229394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:00.905514 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1397/2746] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:00.908109 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a76583-cd9c-414d-869d-3c6b12553f0c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:00.908404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a76583-cd9c-414d-869d-3c6b12553f0c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:00.908741 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a76583-cd9c-414d-869d-3c6b12553f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:00.937046 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-38a76583-cd9c-414d-869d-3c6b12553f0c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:00.938844 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-38a76583-cd9c-414d-869d-3c6b12553f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:00.950638 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1350/2747] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:00.955779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d931ad-cc7f-4721-9d3b-5f6af8229394 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:00.963605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:00.972274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d931ad-cc7f-4721-9d3b-5f6af8229394 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:00.983414 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3919c9-bd86-4a56-b946-b27f18edec95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:00.985205 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3919c9-bd86-4a56-b946-b27f18edec95 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:00.986136 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3919c9-bd86-4a56-b946-b27f18edec95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:00.986531 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3919c9-bd86-4a56-b946-b27f18edec95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:00.987634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3919c9-bd86-4a56-b946-b27f18edec95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:31:00.987943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3919c9-bd86-4a56-b946-b27f18edec95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:00.989914 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3919c9-bd86-4a56-b946-b27f18edec95 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.006743 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1398/2748] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:31:01.014036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a76583-cd9c-414d-869d-3c6b12553f0c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.027335 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a76583-cd9c-414d-869d-3c6b12553f0c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:01.036175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3bf86-b07f-4c71-b61c-64082d358438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:01.037422 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3bf86-b07f-4c71-b61c-64082d358438 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.037970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3bf86-b07f-4c71-b61c-64082d358438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.038510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3bf86-b07f-4c71-b61c-64082d358438 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.039829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3bf86-b07f-4c71-b61c-64082d358438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:01.043744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3bf86-b07f-4c71-b61c-64082d358438 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.089141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1351/2749] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:01 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:01.095757 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3919c9-bd86-4a56-b946-b27f18edec95 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.114309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3919c9-bd86-4a56-b946-b27f18edec95 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:01.214119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505da60-adfc-4edf-81e9-4f4588807b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:01.214939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505da60-adfc-4edf-81e9-4f4588807b0a admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.215290 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505da60-adfc-4edf-81e9-4f4588807b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.215734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505da60-adfc-4edf-81e9-4f4588807b0a admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.216553 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505da60-adfc-4edf-81e9-4f4588807b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:01.218858 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505da60-adfc-4edf-81e9-4f4588807b0a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.247885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1399/2750] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:01 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:01.252571 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3bf86-b07f-4c71-b61c-64082d358438 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.266372 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3bf86-b07f-4c71-b61c-64082d358438 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:01.271501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88a430-fc0f-483f-b0b5-6a8fdfc46a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:01.272196 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88a430-fc0f-483f-b0b5-6a8fdfc46a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.272483 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88a430-fc0f-483f-b0b5-6a8fdfc46a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.272872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88a430-fc0f-483f-b0b5-6a8fdfc46a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.273762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88a430-fc0f-483f-b0b5-6a8fdfc46a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:01.275502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88a430-fc0f-483f-b0b5-6a8fdfc46a0d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.287699 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1352/2751] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:31:01.293276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505da60-adfc-4edf-81e9-4f4588807b0a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.306479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505da60-adfc-4edf-81e9-4f4588807b0a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:01.312021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7d0e2-f268-4f12-8afc-997451084982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:01.312721 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7d0e2-f268-4f12-8afc-997451084982 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.313011 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7d0e2-f268-4f12-8afc-997451084982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.313332 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7d0e2-f268-4f12-8afc-997451084982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.314124 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7d0e2-f268-4f12-8afc-997451084982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:01.315837 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7d0e2-f268-4f12-8afc-997451084982 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.328179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1400/2752] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:01.333579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88a430-fc0f-483f-b0b5-6a8fdfc46a0d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.346034 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88a430-fc0f-483f-b0b5-6a8fdfc46a0d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:01.351415 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6add7f-89de-48c8-ad07-91348f12e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:01.352338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6add7f-89de-48c8-ad07-91348f12e3fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.352616 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6add7f-89de-48c8-ad07-91348f12e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.352898 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6add7f-89de-48c8-ad07-91348f12e3fc admin admin] PATH_INFO: `/v3/projects/6f1396d94f57419aa62828b806464b5c/users/c07d708fe80f46b5bc9108d6fc7227c2/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.371006 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6add7f-89de-48c8-ad07-91348f12e3fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1396d94f57419aa62828b806464b5c, user_id=c07d708fe80f46b5bc9108d6fc7227c2, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:01.373767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6add7f-89de-48c8-ad07-91348f12e3fc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.389856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1353/2753] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:01 2022] PUT /identity/v3/projects/6f1396d94f57419aa62828b806464b5c/users/c07d708fe80f46b5bc9108d6fc7227c2/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:01.402904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eaa76-7f31-4624-8899-0b37db02f24b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.403196 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eaa76-7f31-4624-8899-0b37db02f24b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.403502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eaa76-7f31-4624-8899-0b37db02f24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.444818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8c2eaa76-7f31-4624-8899-0b37db02f24b None None] MFA Rules not processed for user `c07d708fe80f46b5bc9108d6fc7227c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:01.447002 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8c2eaa76-7f31-4624-8899-0b37db02f24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:01.560322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1401/2754] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:01.565341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6add7f-89de-48c8-ad07-91348f12e3fc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.637197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:01.641934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6add7f-89de-48c8-ad07-91348f12e3fc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:01.649437 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16909dc5-4661-49c9-95ca-97c391f6cef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:01.650751 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16909dc5-4661-49c9-95ca-97c391f6cef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.651601 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16909dc5-4661-49c9-95ca-97c391f6cef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.652428 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16909dc5-4661-49c9-95ca-97c391f6cef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.654045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16909dc5-4661-49c9-95ca-97c391f6cef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:01.657970 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16909dc5-4661-49c9-95ca-97c391f6cef6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.702734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1354/2755] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:01 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:01.706925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eaa76-7f31-4624-8899-0b37db02f24b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.722590 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eaa76-7f31-4624-8899-0b37db02f24b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:01.786532 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:01.794283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b29b9-491c-4caa-a4bd-ea223c9afbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:01.795398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b29b9-491c-4caa-a4bd-ea223c9afbcd admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.795954 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b29b9-491c-4caa-a4bd-ea223c9afbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.796314 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b29b9-491c-4caa-a4bd-ea223c9afbcd admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.797295 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b29b9-491c-4caa-a4bd-ea223c9afbcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:01.799440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b29b9-491c-4caa-a4bd-ea223c9afbcd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.831269 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1402/2756] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:01 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:01.837621 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16909dc5-4661-49c9-95ca-97c391f6cef6 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.849553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16909dc5-4661-49c9-95ca-97c391f6cef6 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:01.855148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31335f3a-dea6-4a79-b27b-9c88e1497032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:01.856095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31335f3a-dea6-4a79-b27b-9c88e1497032 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.856382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31335f3a-dea6-4a79-b27b-9c88e1497032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.856663 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31335f3a-dea6-4a79-b27b-9c88e1497032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.857413 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31335f3a-dea6-4a79-b27b-9c88e1497032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:01.859149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31335f3a-dea6-4a79-b27b-9c88e1497032 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.872036 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1355/2757] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:01.878110 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b29b9-491c-4caa-a4bd-ea223c9afbcd admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.897019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b29b9-491c-4caa-a4bd-ea223c9afbcd admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:01.904712 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b4b59-5e2e-46dd-83e9-4dc1d4692998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:01.905772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b4b59-5e2e-46dd-83e9-4dc1d4692998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:01.906113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b4b59-5e2e-46dd-83e9-4dc1d4692998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:01.906434 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b4b59-5e2e-46dd-83e9-4dc1d4692998 admin admin] PATH_INFO: `/v3/projects/f7fdcf7e0d824bf5afa04c82f1827d8a/users/c7f0bd97788a4398b37348061e757503/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:01.927844 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b4b59-5e2e-46dd-83e9-4dc1d4692998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7fdcf7e0d824bf5afa04c82f1827d8a, user_id=c7f0bd97788a4398b37348061e757503, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:01.931673 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b4b59-5e2e-46dd-83e9-4dc1d4692998 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:01.948365 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1403/2758] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:01 2022] PUT /identity/v3/projects/f7fdcf7e0d824bf5afa04c82f1827d8a/users/c7f0bd97788a4398b37348061e757503/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:01.953242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31335f3a-dea6-4a79-b27b-9c88e1497032 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:01.971186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31335f3a-dea6-4a79-b27b-9c88e1497032 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:02.073216 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ea172-bde2-44fc-be92-393ac2d6b0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:02.074468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ea172-bde2-44fc-be92-393ac2d6b0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:02.074948 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ea172-bde2-44fc-be92-393ac2d6b0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:02.075431 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ea172-bde2-44fc-be92-393ac2d6b0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:02.076770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ea172-bde2-44fc-be92-393ac2d6b0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:02.079885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ea172-bde2-44fc-be92-393ac2d6b0b3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:02.096845 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1356/2759] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:02.101363 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b4b59-5e2e-46dd-83e9-4dc1d4692998 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:02.116537 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b4b59-5e2e-46dd-83e9-4dc1d4692998 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:02.122390 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fec6c-b94a-4c2c-9248-5422d28c5ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:02.123324 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fec6c-b94a-4c2c-9248-5422d28c5ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:02.123695 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fec6c-b94a-4c2c-9248-5422d28c5ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:02.123982 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fec6c-b94a-4c2c-9248-5422d28c5ffb admin admin] PATH_INFO: `/v3/projects/f7fdcf7e0d824bf5afa04c82f1827d8a/users/c7f0bd97788a4398b37348061e757503/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:02.126380 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fec6c-b94a-4c2c-9248-5422d28c5ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7fdcf7e0d824bf5afa04c82f1827d8a, user_id=c7f0bd97788a4398b37348061e757503, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:02.129193 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fec6c-b94a-4c2c-9248-5422d28c5ffb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:02.144496 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1404/2760] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:02 2022] PUT /identity/v3/projects/f7fdcf7e0d824bf5afa04c82f1827d8a/users/c7f0bd97788a4398b37348061e757503/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:02.158258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477edf1-6d5a-4a1f-bac4-7e5175a07d89 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:02.159209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477edf1-6d5a-4a1f-bac4-7e5175a07d89 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:02.159724 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477edf1-6d5a-4a1f-bac4-7e5175a07d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:02.233580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a477edf1-6d5a-4a1f-bac4-7e5175a07d89 None None] MFA Rules not processed for user `c7f0bd97788a4398b37348061e757503`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:02.235976 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a477edf1-6d5a-4a1f-bac4-7e5175a07d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:02.362374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1357/2761] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:02.379239 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fec6c-b94a-4c2c-9248-5422d28c5ffb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:02.382779 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-627fec6c-b94a-4c2c-9248-5422d28c5ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:02.479112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fec6c-b94a-4c2c-9248-5422d28c5ffb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:02.484070 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27883a3-2d73-42d4-80ca-5421d1a47c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:02.485019 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27883a3-2d73-42d4-80ca-5421d1a47c47 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:02.485512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27883a3-2d73-42d4-80ca-5421d1a47c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:02.485974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27883a3-2d73-42d4-80ca-5421d1a47c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:02.499998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27883a3-2d73-42d4-80ca-5421d1a47c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:02.501747 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27883a3-2d73-42d4-80ca-5421d1a47c47 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:02.520069 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1405/2762] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:02.662759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477edf1-6d5a-4a1f-bac4-7e5175a07d89 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:02.673975 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477edf1-6d5a-4a1f-bac4-7e5175a07d89 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:02.678528 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fc0b3-fca2-426a-936e-986b50de684f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:02.679337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fc0b3-fca2-426a-936e-986b50de684f service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:02.679929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fc0b3-fca2-426a-936e-986b50de684f service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:02.680207 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fc0b3-fca2-426a-936e-986b50de684f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:02.752698 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:02.770044 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fc0b3-fca2-426a-936e-986b50de684f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:02.773752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fc0b3-fca2-426a-936e-986b50de684f service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:02.795665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1358/2763] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:05.641664 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27883a3-2d73-42d4-80ca-5421d1a47c47 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:05.643694 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a27883a3-2d73-42d4-80ca-5421d1a47c47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:05.701510 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fc0b3-fca2-426a-936e-986b50de684f service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:05.748304 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27883a3-2d73-42d4-80ca-5421d1a47c47 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:05.754820 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:05.756291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:05.757827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:05.758925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] PATH_INFO: `/v3/users/9d078f75a97a478d91bd101f637b9adf` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:05.761490 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d078f75a97a478d91bd101f637b9adf)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:05.765080 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:05.810288 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fc0b3-fca2-426a-936e-986b50de684f service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:05.817508 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:05.818959 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:05.820278 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:05.820461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] PATH_INFO: `/v3/users/c07d708fe80f46b5bc9108d6fc7227c2` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:05.822891 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c07d708fe80f46b5bc9108d6fc7227c2)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:05.825767 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:05.850202 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d078f75a97a478d91bd101f637b9adf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.859037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:05.862523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d078f75a97a478d91bd101f637b9adf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.864512 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d078f75a97a478d91bd101f637b9adf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.880294 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d078f75a97a478d91bd101f637b9adf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.902995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c07d708fe80f46b5bc9108d6fc7227c2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.903818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-88ee7240-cc77-425d-b2b3-d49f91ca2aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d078f75a97a478d91bd101f637b9adf'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.915851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c07d708fe80f46b5bc9108d6fc7227c2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.920144 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1406/2764] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:05 2022] DELETE /identity/v3/users/9d078f75a97a478d91bd101f637b9adf => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:05.920452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c07d708fe80f46b5bc9108d6fc7227c2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.940309 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c07d708fe80f46b5bc9108d6fc7227c2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.941435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c07d708fe80f46b5bc9108d6fc7227c2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:05.965644 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1359/2765] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:05 2022] DELETE /identity/v3/users/c07d708fe80f46b5bc9108d6fc7227c2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:05.971099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a85a-ca3e-4ff1-88cc-9e481d10060f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:05.971538 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a85a-ca3e-4ff1-88cc-9e481d10060f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:05.971986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a85a-ca3e-4ff1-88cc-9e481d10060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:05.995467 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-25a0a85a-ca3e-4ff1-88cc-9e481d10060f None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:05.997431 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-25a0a85a-ca3e-4ff1-88cc-9e481d10060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:06.087229 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1407/2766] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:06.101150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:06.103077 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:06.195853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a85a-ca3e-4ff1-88cc-9e481d10060f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:06.198550 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-25a0a85a-ca3e-4ff1-88cc-9e481d10060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:06.206169 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d4f4d-e319-4b3d-874c-a72729a25430 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:06.212950 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c23da-0cb8-4122-866d-a76078908459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:06.213803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c23da-0cb8-4122-866d-a76078908459 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:06.214258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c23da-0cb8-4122-866d-a76078908459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:06.214614 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c23da-0cb8-4122-866d-a76078908459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:06.227604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a85a-ca3e-4ff1-88cc-9e481d10060f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:06.230904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c23da-0cb8-4122-866d-a76078908459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:06.233085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:06.233646 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c23da-0cb8-4122-866d-a76078908459 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:06.234276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:06.234569 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:06.234847 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] PATH_INFO: `/v3/projects/4194834314a54388a880cbe00c43a79f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:06.236235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4194834314a54388a880cbe00c43a79f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:06.238401 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:06.251837 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1360/2767] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:31:06.320668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4194834314a54388a880cbe00c43a79f', 'request_id': 'req-925ffc42-61d7-4a10-8f04-7014255abe3c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.334021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4194834314a54388a880cbe00c43a79f', 'request_id': 'req-925ffc42-61d7-4a10-8f04-7014255abe3c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.348335 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4194834314a54388a880cbe00c43a79f', 'request_id': 'req-925ffc42-61d7-4a10-8f04-7014255abe3c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.369934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] The token cache is being invalidate because project 4194834314a54388a880cbe00c43a79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:06.370323 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.371135 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.373340 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1408/2768] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:06 2022] DELETE /identity/v3/projects/4194834314a54388a880cbe00c43a79f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:06.378247 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c23da-0cb8-4122-866d-a76078908459 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:06.394180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c23da-0cb8-4122-866d-a76078908459 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:06.408234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:06.502188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:06.504100 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:06.504842 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:06.505777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] PATH_INFO: `/v3/users/9bddb1e3ad7a4ad98f0c25ca4c7b9a1a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:06.509149 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bddb1e3ad7a4ad98f0c25ca4c7b9a1a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:06.513872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:06.610490 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bddb1e3ad7a4ad98f0c25ca4c7b9a1a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.623863 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bddb1e3ad7a4ad98f0c25ca4c7b9a1a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.624885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bddb1e3ad7a4ad98f0c25ca4c7b9a1a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.628274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:06.631394 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:06.648292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ffc42-61d7-4a10-8f04-7014255abe3c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:06.651152 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bddb1e3ad7a4ad98f0c25ca4c7b9a1a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.652189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bddb1e3ad7a4ad98f0c25ca4c7b9a1a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.657317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:06.659034 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:06.659634 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:06.660961 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] PATH_INFO: `/v3/projects/6f1396d94f57419aa62828b806464b5c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:06.665987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f1396d94f57419aa62828b806464b5c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:06.669985 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:06.680387 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1361/2769] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:06 2022] DELETE /identity/v3/users/9bddb1e3ad7a4ad98f0c25ca4c7b9a1a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:06.735628 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f1396d94f57419aa62828b806464b5c', 'request_id': 'req-9f6826e6-1c42-4b02-9165-251c1b04d606'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.737754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:06.751761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f1396d94f57419aa62828b806464b5c', 'request_id': 'req-9f6826e6-1c42-4b02-9165-251c1b04d606'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.766633 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f1396d94f57419aa62828b806464b5c', 'request_id': 'req-9f6826e6-1c42-4b02-9165-251c1b04d606'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.784562 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] The token cache is being invalidate because project 6f1396d94f57419aa62828b806464b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:06.785112 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.786261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:06.790883 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1409/2770] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:06 2022] DELETE /identity/v3/projects/6f1396d94f57419aa62828b806464b5c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:06.798449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:06.801115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:06.922392 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:06.926285 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:06.937592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366f8d8-6576-4f91-8b08-5ca5a97821c9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:07.012986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:07.041342 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6826e6-1c42-4b02-9165-251c1b04d606 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:07.046402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:07.048197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.048847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.049233 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:07.049382 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] PATH_INFO: `/v3/users/c7f0bd97788a4398b37348061e757503` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.050681 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.051154 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.051591 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7f0bd97788a4398b37348061e757503)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:07.051718 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] PATH_INFO: `/v3/projects/95df364cf8a44185864291e4a856f3d2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.054357 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95df364cf8a44185864291e4a856f3d2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:07.056052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:07.057879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:07.141236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95df364cf8a44185864291e4a856f3d2', 'request_id': 'req-49a7a49f-9d91-4ec1-8786-a6aab0d52311'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.150134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7f0bd97788a4398b37348061e757503'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.152872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95df364cf8a44185864291e4a856f3d2', 'request_id': 'req-49a7a49f-9d91-4ec1-8786-a6aab0d52311'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.159806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7f0bd97788a4398b37348061e757503'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.160776 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7f0bd97788a4398b37348061e757503'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.167318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95df364cf8a44185864291e4a856f3d2', 'request_id': 'req-49a7a49f-9d91-4ec1-8786-a6aab0d52311'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.179686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7f0bd97788a4398b37348061e757503'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.180513 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e134db13-d789-47d3-933a-d098b2f8f8ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7f0bd97788a4398b37348061e757503'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.189919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] The token cache is being invalidate because project 95df364cf8a44185864291e4a856f3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:07.190439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.191575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-49a7a49f-9d91-4ec1-8786-a6aab0d52311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.194615 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1410/2771] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:06 2022] DELETE /identity/v3/projects/95df364cf8a44185864291e4a856f3d2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:07.199501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1362/2772] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:06 2022] DELETE /identity/v3/users/c7f0bd97788a4398b37348061e757503 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:07.201495 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1af9f-c8af-4e0e-9929-04b0e2cc2ad7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.202130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1af9f-c8af-4e0e-9929-04b0e2cc2ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.202734 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1af9f-c8af-4e0e-9929-04b0e2cc2ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.232777 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d5c1af9f-c8af-4e0e-9929-04b0e2cc2ad7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:07.235118 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d5c1af9f-c8af-4e0e-9929-04b0e2cc2ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:07.359443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1411/2773] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:07.369500 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd1443-f620-47a9-aeee-ba49fd21c510 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.370052 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd1443-f620-47a9-aeee-ba49fd21c510 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.370550 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd1443-f620-47a9-aeee-ba49fd21c510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.403738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ccdd1443-f620-47a9-aeee-ba49fd21c510 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:07.405907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ccdd1443-f620-47a9-aeee-ba49fd21c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:07.423743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1363/2774] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:07.426109 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1af9f-c8af-4e0e-9929-04b0e2cc2ad7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:07.445014 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1af9f-c8af-4e0e-9929-04b0e2cc2ad7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:07.453662 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dec5a8-c5da-4d94-8eae-f9f89af0d4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:07.455500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dec5a8-c5da-4d94-8eae-f9f89af0d4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.455873 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dec5a8-c5da-4d94-8eae-f9f89af0d4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.456468 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dec5a8-c5da-4d94-8eae-f9f89af0d4b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.458523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dec5a8-c5da-4d94-8eae-f9f89af0d4b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:31:07.459111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dec5a8-c5da-4d94-8eae-f9f89af0d4b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:07.462691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dec5a8-c5da-4d94-8eae-f9f89af0d4b5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:07.467801 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:07.494791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1412/2775] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:31:07.499969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd1443-f620-47a9-aeee-ba49fd21c510 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:07.513934 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd1443-f620-47a9-aeee-ba49fd21c510 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:07.519848 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8446b5-d616-4880-9cc2-5d44ae00ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:07.520474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8446b5-d616-4880-9cc2-5d44ae00ba20 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.520889 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8446b5-d616-4880-9cc2-5d44ae00ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.521233 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8446b5-d616-4880-9cc2-5d44ae00ba20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.522073 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8446b5-d616-4880-9cc2-5d44ae00ba20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:07.525784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8446b5-d616-4880-9cc2-5d44ae00ba20 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:07.562148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1364/2776] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:07 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:07.567504 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dec5a8-c5da-4d94-8eae-f9f89af0d4b5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:07.587393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dec5a8-c5da-4d94-8eae-f9f89af0d4b5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:07.673065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8446b5-d616-4880-9cc2-5d44ae00ba20 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:07.676720 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2f8446b5-d616-4880-9cc2-5d44ae00ba20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:07.681048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6a66-6003-406f-b703-abed13b95bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:07.682533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6a66-6003-406f-b703-abed13b95bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.683167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6a66-6003-406f-b703-abed13b95bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.683939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6a66-6003-406f-b703-abed13b95bdf admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.685454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a6a66-6003-406f-b703-abed13b95bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:07.689563 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a6a66-6003-406f-b703-abed13b95bdf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:07.705630 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8446b5-d616-4880-9cc2-5d44ae00ba20 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:07.715917 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:07.717452 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.718018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.718511 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] PATH_INFO: `/v3/projects/f7fdcf7e0d824bf5afa04c82f1827d8a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.720865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7fdcf7e0d824bf5afa04c82f1827d8a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:07.727861 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1413/2777] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:07 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:07.728164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:07.729310 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6a66-6003-406f-b703-abed13b95bdf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:07.741188 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6a66-6003-406f-b703-abed13b95bdf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:07.747574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c2b83-7285-4d9f-a9d2-8ef016a40e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:07.748293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c2b83-7285-4d9f-a9d2-8ef016a40e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.748570 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c2b83-7285-4d9f-a9d2-8ef016a40e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.748857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c2b83-7285-4d9f-a9d2-8ef016a40e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.749613 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c2b83-7285-4d9f-a9d2-8ef016a40e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:07.751364 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c2b83-7285-4d9f-a9d2-8ef016a40e13 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:07.769539 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1414/2778] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:07.778015 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c2b83-7285-4d9f-a9d2-8ef016a40e13 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:07.790904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c2b83-7285-4d9f-a9d2-8ef016a40e13 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:07.796376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7fdcf7e0d824bf5afa04c82f1827d8a', 'request_id': 'req-9484db3f-7759-494c-b139-2d26623c1e76'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.806471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7fdcf7e0d824bf5afa04c82f1827d8a', 'request_id': 'req-9484db3f-7759-494c-b139-2d26623c1e76'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.814048 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7fdcf7e0d824bf5afa04c82f1827d8a', 'request_id': 'req-9484db3f-7759-494c-b139-2d26623c1e76'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.832176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] The token cache is being invalidate because project f7fdcf7e0d824bf5afa04c82f1827d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:07.832609 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.833801 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9484db3f-7759-494c-b139-2d26623c1e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:07.836520 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1365/2779] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:07 2022] DELETE /identity/v3/projects/f7fdcf7e0d824bf5afa04c82f1827d8a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:07.842385 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe82bf2-4d7c-4224-882e-0d3f5b92088a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.842851 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe82bf2-4d7c-4224-882e-0d3f5b92088a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.843242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe82bf2-4d7c-4224-882e-0d3f5b92088a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.868707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-afe82bf2-4d7c-4224-882e-0d3f5b92088a None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:07.870558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-afe82bf2-4d7c-4224-882e-0d3f5b92088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:07.882895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c754c0-9484-4570-8e37-9c035f957235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:07.883746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:07.883964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c754c0-9484-4570-8e37-9c035f957235 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.884210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c754c0-9484-4570-8e37-9c035f957235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.884666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c754c0-9484-4570-8e37-9c035f957235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.886237 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c754c0-9484-4570-8e37-9c035f957235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:07.889210 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c754c0-9484-4570-8e37-9c035f957235 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:07.902353 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1415/2780] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:31:07.908932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c754c0-9484-4570-8e37-9c035f957235 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:07.911826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a1c754c0-9484-4570-8e37-9c035f957235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:07.935441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c754c0-9484-4570-8e37-9c035f957235 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:07.943317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123b1d5-7721-4249-a9d3-60d74e31c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:07.945456 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123b1d5-7721-4249-a9d3-60d74e31c092 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.945996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123b1d5-7721-4249-a9d3-60d74e31c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.946716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123b1d5-7721-4249-a9d3-60d74e31c092 admin admin] PATH_INFO: `/v3/projects/f202bbd88dc54176b2a0712971db5ac0/users/655958c6a4f041b3b5648f36c394e427/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.969357 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123b1d5-7721-4249-a9d3-60d74e31c092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f202bbd88dc54176b2a0712971db5ac0, user_id=655958c6a4f041b3b5648f36c394e427, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:07.972980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123b1d5-7721-4249-a9d3-60d74e31c092 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:07.980286 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1366/2781] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:07.988992 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f2aac-16ac-409f-9494-2dbc62b3810e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:07.989361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f2aac-16ac-409f-9494-2dbc62b3810e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:07.989587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f2aac-16ac-409f-9494-2dbc62b3810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:07.992241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1416/2782] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:07 2022] PUT /identity/v3/projects/f202bbd88dc54176b2a0712971db5ac0/users/655958c6a4f041b3b5648f36c394e427/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:08.016897 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-061f2aac-16ac-409f-9494-2dbc62b3810e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:08.018243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-061f2aac-16ac-409f-9494-2dbc62b3810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:08.086885 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae68da9-bf64-41ea-aef5-343f16f8c068 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.087322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae68da9-bf64-41ea-aef5-343f16f8c068 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.087780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae68da9-bf64-41ea-aef5-343f16f8c068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.123174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cae68da9-bf64-41ea-aef5-343f16f8c068 None None] MFA Rules not processed for user `655958c6a4f041b3b5648f36c394e427`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:08.125561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cae68da9-bf64-41ea-aef5-343f16f8c068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:08.130296 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1367/2783] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:31:08.132157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f2aac-16ac-409f-9494-2dbc62b3810e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.147144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f2aac-16ac-409f-9494-2dbc62b3810e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.153496 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7c77b-5666-440d-bea4-712cd64a133c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.154384 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7c77b-5666-440d-bea4-712cd64a133c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.154775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7c77b-5666-440d-bea4-712cd64a133c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.155221 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7c77b-5666-440d-bea4-712cd64a133c admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.156506 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7c77b-5666-440d-bea4-712cd64a133c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:31:08.157022 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7c77b-5666-440d-bea4-712cd64a133c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.159146 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7c77b-5666-440d-bea4-712cd64a133c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.176085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1368/2784] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:31:08.181910 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:08.182266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7c77b-5666-440d-bea4-712cd64a133c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.198758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7c77b-5666-440d-bea4-712cd64a133c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.207082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379f376-41c2-4836-b39d-4672c99bfeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.208231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379f376-41c2-4836-b39d-4672c99bfeb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.208714 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379f376-41c2-4836-b39d-4672c99bfeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.209200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379f376-41c2-4836-b39d-4672c99bfeb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.210664 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379f376-41c2-4836-b39d-4672c99bfeb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.215547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379f376-41c2-4836-b39d-4672c99bfeb4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.234867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1417/2785] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:31:08.248281 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae68da9-bf64-41ea-aef5-343f16f8c068 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.250039 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cae68da9-bf64-41ea-aef5-343f16f8c068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:08.258955 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1369/2786] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:08 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:08.263759 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379f376-41c2-4836-b39d-4672c99bfeb4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.331877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae68da9-bf64-41ea-aef5-343f16f8c068 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.356703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379f376-41c2-4836-b39d-4672c99bfeb4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.365192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1544ff-906e-410b-a589-e50798f8b8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.367605 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1544ff-906e-410b-a589-e50798f8b8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.368117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1544ff-906e-410b-a589-e50798f8b8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.368625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1544ff-906e-410b-a589-e50798f8b8b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.370057 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1544ff-906e-410b-a589-e50798f8b8b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.373967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1544ff-906e-410b-a589-e50798f8b8b8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.395925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26f368-ea3c-44ce-a02f-277c55a18c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.396614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26f368-ea3c-44ce-a02f-277c55a18c74 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.396898 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26f368-ea3c-44ce-a02f-277c55a18c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.397346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26f368-ea3c-44ce-a02f-277c55a18c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.412344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1370/2787] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:08 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:08.415543 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1544ff-906e-410b-a589-e50798f8b8b8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.431234 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1544ff-906e-410b-a589-e50798f8b8b8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.437157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48efe48-c2ce-4be9-94b7-6d4a450948f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.438253 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48efe48-c2ce-4be9-94b7-6d4a450948f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.438632 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48efe48-c2ce-4be9-94b7-6d4a450948f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.438978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48efe48-c2ce-4be9-94b7-6d4a450948f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.440003 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48efe48-c2ce-4be9-94b7-6d4a450948f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.442083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48efe48-c2ce-4be9-94b7-6d4a450948f3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.445402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:08.461303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1371/2788] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:08.462232 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48efe48-c2ce-4be9-94b7-6d4a450948f3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.472839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48efe48-c2ce-4be9-94b7-6d4a450948f3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.478371 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb8c68-039e-4b1c-bc50-9f4395eebb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.479125 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb8c68-039e-4b1c-bc50-9f4395eebb18 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.479418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb8c68-039e-4b1c-bc50-9f4395eebb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.479701 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb8c68-039e-4b1c-bc50-9f4395eebb18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.480459 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb8c68-039e-4b1c-bc50-9f4395eebb18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.482331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26f368-ea3c-44ce-a02f-277c55a18c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.482560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb8c68-039e-4b1c-bc50-9f4395eebb18 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.484068 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26f368-ea3c-44ce-a02f-277c55a18c74 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.495203 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1372/2789] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:08.497892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1418/2790] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:31:08.499676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb8c68-039e-4b1c-bc50-9f4395eebb18 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.511168 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb8c68-039e-4b1c-bc50-9f4395eebb18 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.519736 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd9506-b513-4712-866f-450751c71932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.521449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd9506-b513-4712-866f-450751c71932 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.521991 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd9506-b513-4712-866f-450751c71932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.522981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd9506-b513-4712-866f-450751c71932 admin admin] PATH_INFO: `/v3/projects/acf76bd8ae0e434c8c996c6a16e8c1b8/users/1678c53430094d17b174f9f0d03c8ad4/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.524560 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26f368-ea3c-44ce-a02f-277c55a18c74 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.539045 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26f368-ea3c-44ce-a02f-277c55a18c74 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.544938 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.546043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd9506-b513-4712-866f-450751c71932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf76bd8ae0e434c8c996c6a16e8c1b8, user_id=1678c53430094d17b174f9f0d03c8ad4, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.546291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.546760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.547143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] PATH_INFO: `/v3/users/655958c6a4f041b3b5648f36c394e427` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.548629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=655958c6a4f041b3b5648f36c394e427)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.549068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd9506-b513-4712-866f-450751c71932 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.550895 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.566814 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1373/2791] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:08 2022] PUT /identity/v3/projects/acf76bd8ae0e434c8c996c6a16e8c1b8/users/1678c53430094d17b174f9f0d03c8ad4/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:08.584447 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c533f-dc14-4482-8770-3fdd5c995794 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.584769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c533f-dc14-4482-8770-3fdd5c995794 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.585580 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c533f-dc14-4482-8770-3fdd5c995794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.617013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:08.620180 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-536c533f-dc14-4482-8770-3fdd5c995794 None None] MFA Rules not processed for user `1678c53430094d17b174f9f0d03c8ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:08.621435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-536c533f-dc14-4482-8770-3fdd5c995794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:08.628119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655958c6a4f041b3b5648f36c394e427'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:08.639731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655958c6a4f041b3b5648f36c394e427'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:08.640646 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655958c6a4f041b3b5648f36c394e427'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:08.650819 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655958c6a4f041b3b5648f36c394e427'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:08.666744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c0669491-5b8c-448e-96d3-aac77b06af07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655958c6a4f041b3b5648f36c394e427'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:08.674007 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1419/2792] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:08 2022] DELETE /identity/v3/users/655958c6a4f041b3b5648f36c394e427 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:08.678913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c316f-931c-419a-8083-f0b9d65b2a5e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.679677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c316f-931c-419a-8083-f0b9d65b2a5e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.680162 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c316f-931c-419a-8083-f0b9d65b2a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.700429 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a04c316f-931c-419a-8083-f0b9d65b2a5e None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:08.701731 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a04c316f-931c-419a-8083-f0b9d65b2a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:08.748225 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1374/2793] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:31:08.755555 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c533f-dc14-4482-8770-3fdd5c995794 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.759065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-536c533f-dc14-4482-8770-3fdd5c995794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:08.809013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1420/2794] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:08.815987 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c316f-931c-419a-8083-f0b9d65b2a5e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.817906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a04c316f-931c-419a-8083-f0b9d65b2a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:08.852723 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c533f-dc14-4482-8770-3fdd5c995794 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.859080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fdfa6-7a40-41fc-a49a-a1c74ed7abdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.860016 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fdfa6-7a40-41fc-a49a-a1c74ed7abdd admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.860472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fdfa6-7a40-41fc-a49a-a1c74ed7abdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.860830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fdfa6-7a40-41fc-a49a-a1c74ed7abdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.861834 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fdfa6-7a40-41fc-a49a-a1c74ed7abdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.864364 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fdfa6-7a40-41fc-a49a-a1c74ed7abdd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.903633 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1375/2795] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:08 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:08.903925 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c316f-931c-419a-8083-f0b9d65b2a5e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.905080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fdfa6-7a40-41fc-a49a-a1c74ed7abdd admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:08.963877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:08.979770 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f32208-4877-49cf-8e3c-67f328b998d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.980852 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f32208-4877-49cf-8e3c-67f328b998d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.981334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f32208-4877-49cf-8e3c-67f328b998d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.981740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f32208-4877-49cf-8e3c-67f328b998d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:08.993577 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fdfa6-7a40-41fc-a49a-a1c74ed7abdd admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:08.995046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f32208-4877-49cf-8e3c-67f328b998d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:08.996868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f32208-4877-49cf-8e3c-67f328b998d2 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:08.998859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3729a-1c5f-447a-89bd-93139f8924c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:08.999457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3729a-1c5f-447a-89bd-93139f8924c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:08.999753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3729a-1c5f-447a-89bd-93139f8924c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:08.999997 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3729a-1c5f-447a-89bd-93139f8924c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.000737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3729a-1c5f-447a-89bd-93139f8924c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:09.002872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3729a-1c5f-447a-89bd-93139f8924c9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:09.014685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1421/2796] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:09.032229 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1376/2797] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:08 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:09.038228 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f32208-4877-49cf-8e3c-67f328b998d2 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:09.057502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f32208-4877-49cf-8e3c-67f328b998d2 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:09.065689 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666291ba-ba7b-4883-ac69-c195c7345552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:09.066486 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666291ba-ba7b-4883-ac69-c195c7345552 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.066771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666291ba-ba7b-4883-ac69-c195c7345552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.067048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666291ba-ba7b-4883-ac69-c195c7345552 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.067879 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666291ba-ba7b-4883-ac69-c195c7345552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:09.070915 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666291ba-ba7b-4883-ac69-c195c7345552 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:09.093148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1422/2798] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:09.094994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3729a-1c5f-447a-89bd-93139f8924c9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:09.109328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3729a-1c5f-447a-89bd-93139f8924c9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:09.115269 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f10b-80cf-4983-910e-3367583f1513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:09.116793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f10b-80cf-4983-910e-3367583f1513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.117037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f10b-80cf-4983-910e-3367583f1513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.117346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f10b-80cf-4983-910e-3367583f1513 admin admin] PATH_INFO: `/v3/projects/f1faed53dd6541ecb1717e855e5d7f32/users/5c4599b4f0f741adb3183f3495f84ddf/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.141758 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4f10b-80cf-4983-910e-3367583f1513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1faed53dd6541ecb1717e855e5d7f32, user_id=5c4599b4f0f741adb3183f3495f84ddf, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:09.145272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4f10b-80cf-4983-910e-3367583f1513 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:09.166192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1377/2799] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:09 2022] PUT /identity/v3/projects/f1faed53dd6541ecb1717e855e5d7f32/users/5c4599b4f0f741adb3183f3495f84ddf/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:09.171501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666291ba-ba7b-4883-ac69-c195c7345552 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:09.185668 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666291ba-ba7b-4883-ac69-c195c7345552 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:09.191400 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:09.295983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fded9e3-d2d9-410a-8845-449a47270532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:09.297092 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded9e3-d2d9-410a-8845-449a47270532 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.297611 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded9e3-d2d9-410a-8845-449a47270532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.298184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded9e3-d2d9-410a-8845-449a47270532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.300009 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fded9e3-d2d9-410a-8845-449a47270532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:09.303008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fded9e3-d2d9-410a-8845-449a47270532 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:09.324597 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1423/2800] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:09.329624 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f10b-80cf-4983-910e-3367583f1513 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:09.350055 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f10b-80cf-4983-910e-3367583f1513 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:09.359197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3df42-27f4-49aa-811b-350a931efb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:09.360833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3df42-27f4-49aa-811b-350a931efb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.361337 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3df42-27f4-49aa-811b-350a931efb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.361879 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3df42-27f4-49aa-811b-350a931efb00 admin admin] PATH_INFO: `/v3/projects/f1faed53dd6541ecb1717e855e5d7f32/users/5c4599b4f0f741adb3183f3495f84ddf/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.365779 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3df42-27f4-49aa-811b-350a931efb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1faed53dd6541ecb1717e855e5d7f32, user_id=5c4599b4f0f741adb3183f3495f84ddf, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:09.371968 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3df42-27f4-49aa-811b-350a931efb00 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:09.402023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1378/2801] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:09 2022] PUT /identity/v3/projects/f1faed53dd6541ecb1717e855e5d7f32/users/5c4599b4f0f741adb3183f3495f84ddf/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:09.415013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680146d-132e-4c82-a86e-d139c9cd4f4d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.415367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680146d-132e-4c82-a86e-d139c9cd4f4d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.415533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680146d-132e-4c82-a86e-d139c9cd4f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.426791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3df42-27f4-49aa-811b-350a931efb00 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:09.430365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b6c3df42-27f4-49aa-811b-350a931efb00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:09.471264 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5680146d-132e-4c82-a86e-d139c9cd4f4d None None] MFA Rules not processed for user `5c4599b4f0f741adb3183f3495f84ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:09.473830 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5680146d-132e-4c82-a86e-d139c9cd4f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:09.568753 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3df42-27f4-49aa-811b-350a931efb00 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:09.576592 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:09.577492 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.577808 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.578082 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] PATH_INFO: `/v3/projects/f202bbd88dc54176b2a0712971db5ac0` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.579985 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f202bbd88dc54176b2a0712971db5ac0)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:09.582063 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:09.620855 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1424/2802] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:31:09.633690 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680146d-132e-4c82-a86e-d139c9cd4f4d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:09.635217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5680146d-132e-4c82-a86e-d139c9cd4f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:09.650341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f202bbd88dc54176b2a0712971db5ac0', 'request_id': 'req-043746d8-d475-4b1c-8142-1746f81b8d27'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:09.659393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f202bbd88dc54176b2a0712971db5ac0', 'request_id': 'req-043746d8-d475-4b1c-8142-1746f81b8d27'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:09.667788 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f202bbd88dc54176b2a0712971db5ac0', 'request_id': 'req-043746d8-d475-4b1c-8142-1746f81b8d27'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:09.682176 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] The token cache is being invalidate because project f202bbd88dc54176b2a0712971db5ac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:09.682481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:09.683184 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-043746d8-d475-4b1c-8142-1746f81b8d27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:09.685141 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1379/2803] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:09 2022] DELETE /identity/v3/projects/f202bbd88dc54176b2a0712971db5ac0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:09.692173 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187e9b-ae68-4028-9789-0984382be84c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.692881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187e9b-ae68-4028-9789-0984382be84c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.692881 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187e9b-ae68-4028-9789-0984382be84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.706075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680146d-132e-4c82-a86e-d139c9cd4f4d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:09.723361 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e6187e9b-ae68-4028-9789-0984382be84c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:09.726272 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e6187e9b-ae68-4028-9789-0984382be84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:09.756553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:09.760437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:09.776442 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0110cf1-64d8-4440-a7b6-596a1d1b2fbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:09.777498 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0110cf1-64d8-4440-a7b6-596a1d1b2fbe service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.777994 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0110cf1-64d8-4440-a7b6-596a1d1b2fbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.778464 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0110cf1-64d8-4440-a7b6-596a1d1b2fbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.782379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b0110cf1-64d8-4440-a7b6-596a1d1b2fbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:09.840922 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1380/2804] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:31:09.850159 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d03524-ee6c-415e-93c0-be8223c938d7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.851175 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d03524-ee6c-415e-93c0-be8223c938d7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.851478 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d03524-ee6c-415e-93c0-be8223c938d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.872155 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-39d03524-ee6c-415e-93c0-be8223c938d7 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:09.873393 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-39d03524-ee6c-415e-93c0-be8223c938d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:09.882164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1381/2805] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:31:09.886584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d03524-ee6c-415e-93c0-be8223c938d7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:09.895474 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0110cf1-64d8-4440-a7b6-596a1d1b2fbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:09.897205 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d03524-ee6c-415e-93c0-be8223c938d7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:09.898952 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0110cf1-64d8-4440-a7b6-596a1d1b2fbe service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:09.904517 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba43c4a-bac1-495f-9dcd-6c471c10725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:09.905399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba43c4a-bac1-495f-9dcd-6c471c10725f admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.905856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba43c4a-bac1-495f-9dcd-6c471c10725f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.906318 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba43c4a-bac1-495f-9dcd-6c471c10725f admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.907568 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba43c4a-bac1-495f-9dcd-6c471c10725f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:31:09.907974 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba43c4a-bac1-495f-9dcd-6c471c10725f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:09.910006 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba43c4a-bac1-495f-9dcd-6c471c10725f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:09.918479 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1425/2806] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:09.929515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1382/2807] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 14:31:09.934868 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0110cf1-64d8-4440-a7b6-596a1d1b2fbe service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:09.954860 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0110cf1-64d8-4440-a7b6-596a1d1b2fbe service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:09.963726 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f710eb-b549-43da-b72f-e547579eb68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:09.964935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f710eb-b549-43da-b72f-e547579eb68f admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:09.965923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f710eb-b549-43da-b72f-e547579eb68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:09.966506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f710eb-b549-43da-b72f-e547579eb68f admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:09.967872 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f710eb-b549-43da-b72f-e547579eb68f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:09.971771 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f710eb-b549-43da-b72f-e547579eb68f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:10.016369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1426/2808] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:09 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:10.021148 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba43c4a-bac1-495f-9dcd-6c471c10725f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:10.035668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba43c4a-bac1-495f-9dcd-6c471c10725f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:10.116821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5ae91-1637-424e-baa3-7041e3189550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:10.117686 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5ae91-1637-424e-baa3-7041e3189550 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:10.118188 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5ae91-1637-424e-baa3-7041e3189550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:10.118606 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5ae91-1637-424e-baa3-7041e3189550 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:10.119516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5ae91-1637-424e-baa3-7041e3189550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:10.122362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5ae91-1637-424e-baa3-7041e3189550 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:10.152000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1383/2809] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:10 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:10.157341 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f710eb-b549-43da-b72f-e547579eb68f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:10.175387 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f710eb-b549-43da-b72f-e547579eb68f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:10.183509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4ec4c-4a7b-46c4-995e-f702ccc510a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:10.184534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4ec4c-4a7b-46c4-995e-f702ccc510a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:10.184973 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4ec4c-4a7b-46c4-995e-f702ccc510a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:10.185439 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4ec4c-4a7b-46c4-995e-f702ccc510a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:10.186854 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4ec4c-4a7b-46c4-995e-f702ccc510a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:10.189645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4ec4c-4a7b-46c4-995e-f702ccc510a3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:10.215614 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1427/2810] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:10.217299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5ae91-1637-424e-baa3-7041e3189550 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:10.235978 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5ae91-1637-424e-baa3-7041e3189550 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:10.244376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337d8fe-5440-49f0-84a2-49dab2c97af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:10.245403 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337d8fe-5440-49f0-84a2-49dab2c97af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:10.245902 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337d8fe-5440-49f0-84a2-49dab2c97af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:10.246354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337d8fe-5440-49f0-84a2-49dab2c97af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:10.247558 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337d8fe-5440-49f0-84a2-49dab2c97af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:10.250293 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337d8fe-5440-49f0-84a2-49dab2c97af8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:10.274277 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1384/2811] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:10.275588 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4ec4c-4a7b-46c4-995e-f702ccc510a3 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:10.287211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4ec4c-4a7b-46c4-995e-f702ccc510a3 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:10.293781 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc9a35-649b-498e-844f-35992eb3fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:10.295006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc9a35-649b-498e-844f-35992eb3fb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:10.295458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc9a35-649b-498e-844f-35992eb3fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:10.295950 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc9a35-649b-498e-844f-35992eb3fb8a admin admin] PATH_INFO: `/v3/projects/8963278084b2491799c6c5e403b5cfcc/users/c795167b3e194db6925da938d9572912/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:10.315529 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc9a35-649b-498e-844f-35992eb3fb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8963278084b2491799c6c5e403b5cfcc, user_id=c795167b3e194db6925da938d9572912, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:10.318462 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc9a35-649b-498e-844f-35992eb3fb8a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:10.336050 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1428/2812] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:10 2022] PUT /identity/v3/projects/8963278084b2491799c6c5e403b5cfcc/users/c795167b3e194db6925da938d9572912/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:10.341462 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb9ff2-a186-41a9-8e2d-74a8cdec1420 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:10.341821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb9ff2-a186-41a9-8e2d-74a8cdec1420 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:10.342109 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb9ff2-a186-41a9-8e2d-74a8cdec1420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:10.363791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-10eb9ff2-a186-41a9-8e2d-74a8cdec1420 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:10.364932 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-10eb9ff2-a186-41a9-8e2d-74a8cdec1420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:10.453770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1385/2813] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:10.464788 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc9a35-649b-498e-844f-35992eb3fb8a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:10.467558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-19dc9a35-649b-498e-844f-35992eb3fb8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:10.578794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc9a35-649b-498e-844f-35992eb3fb8a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:10.582752 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d88f59-8f4d-478b-8797-652b9ad16362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:10.583424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d88f59-8f4d-478b-8797-652b9ad16362 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:10.583711 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d88f59-8f4d-478b-8797-652b9ad16362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:10.583990 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d88f59-8f4d-478b-8797-652b9ad16362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:10.594857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d88f59-8f4d-478b-8797-652b9ad16362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:10.596351 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d88f59-8f4d-478b-8797-652b9ad16362 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:10.610298 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1429/2814] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:10.982892 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb9ff2-a186-41a9-8e2d-74a8cdec1420 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:10.984773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-10eb9ff2-a186-41a9-8e2d-74a8cdec1420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:11.000882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb9ff2-a186-41a9-8e2d-74a8cdec1420 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:11.006752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:11.007062 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:11.007332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:11.007623 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] PATH_INFO: `/v3/users/1678c53430094d17b174f9f0d03c8ad4` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:11.008989 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1678c53430094d17b174f9f0d03c8ad4)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:11.011136 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:11.030331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:11.100108 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1678c53430094d17b174f9f0d03c8ad4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:11.114981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1678c53430094d17b174f9f0d03c8ad4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:11.117993 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1678c53430094d17b174f9f0d03c8ad4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:11.149466 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1678c53430094d17b174f9f0d03c8ad4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:11.150388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1678c53430094d17b174f9f0d03c8ad4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:11.173589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1386/2815] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:10 2022] DELETE /identity/v3/users/1678c53430094d17b174f9f0d03c8ad4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:11.179834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e165aa-fc37-4c26-bd2c-06ec44e299e9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:11.180393 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e165aa-fc37-4c26-bd2c-06ec44e299e9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:11.180601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e165aa-fc37-4c26-bd2c-06ec44e299e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:11.200130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-52e165aa-fc37-4c26-bd2c-06ec44e299e9 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:11.201469 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-52e165aa-fc37-4c26-bd2c-06ec44e299e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:11.245458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:11.287131 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1430/2816] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:11.298503 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:11.300354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:11.409269 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4431326-f87c-46e8-87c2-b84ddced04fd admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:11.414250 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26778498-11a7-4e22-af07-8b400a31d425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:11.415799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26778498-11a7-4e22-af07-8b400a31d425 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:11.416291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26778498-11a7-4e22-af07-8b400a31d425 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:11.416772 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26778498-11a7-4e22-af07-8b400a31d425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:11.434475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26778498-11a7-4e22-af07-8b400a31d425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:11.436984 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26778498-11a7-4e22-af07-8b400a31d425 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:11.458259 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1387/2817] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:31:11.899996 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e165aa-fc37-4c26-bd2c-06ec44e299e9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:11.901511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-52e165aa-fc37-4c26-bd2c-06ec44e299e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:11.923651 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e165aa-fc37-4c26-bd2c-06ec44e299e9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:11.932993 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:11.934791 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:11.935299 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:11.935686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] PATH_INFO: `/v3/projects/acf76bd8ae0e434c8c996c6a16e8c1b8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:11.937929 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf76bd8ae0e434c8c996c6a16e8c1b8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:11.941802 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:12.038896 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf76bd8ae0e434c8c996c6a16e8c1b8', 'request_id': 'req-97bf9834-7619-400c-9b64-e3f50fa1a93b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.051111 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf76bd8ae0e434c8c996c6a16e8c1b8', 'request_id': 'req-97bf9834-7619-400c-9b64-e3f50fa1a93b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.064152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf76bd8ae0e434c8c996c6a16e8c1b8', 'request_id': 'req-97bf9834-7619-400c-9b64-e3f50fa1a93b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.083180 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] The token cache is being invalidate because project acf76bd8ae0e434c8c996c6a16e8c1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:12.083836 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.084834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.088189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1431/2818] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:11 2022] DELETE /identity/v3/projects/acf76bd8ae0e434c8c996c6a16e8c1b8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:12.091752 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26778498-11a7-4e22-af07-8b400a31d425 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:12.104258 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26778498-11a7-4e22-af07-8b400a31d425 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:12.184516 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:12.185547 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:12.185946 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:12.186290 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] PATH_INFO: `/v3/users/5c4599b4f0f741adb3183f3495f84ddf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:12.187771 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c4599b4f0f741adb3183f3495f84ddf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:12.189856 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:12.252140 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:12.276045 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c4599b4f0f741adb3183f3495f84ddf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.292349 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c4599b4f0f741adb3183f3495f84ddf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.293791 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c4599b4f0f741adb3183f3495f84ddf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.326764 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c4599b4f0f741adb3183f3495f84ddf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.327569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4af96a88-fab3-4a8f-b052-fd484e2ca313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c4599b4f0f741adb3183f3495f84ddf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:12.348749 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1388/2819] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:12 2022] DELETE /identity/v3/users/5c4599b4f0f741adb3183f3495f84ddf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:12.956904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:12.960384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:12.979831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf9834-7619-400c-9b64-e3f50fa1a93b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:13.122235 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:13.123344 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:13.123610 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:13.123892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] PATH_INFO: `/v3/projects/f1faed53dd6541ecb1717e855e5d7f32` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:13.125986 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1faed53dd6541ecb1717e855e5d7f32)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:13.128402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:13.202453 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1faed53dd6541ecb1717e855e5d7f32', 'request_id': 'req-3263777c-7981-4449-b326-d7e007143e61'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:13.214203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1faed53dd6541ecb1717e855e5d7f32', 'request_id': 'req-3263777c-7981-4449-b326-d7e007143e61'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:13.223425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1faed53dd6541ecb1717e855e5d7f32', 'request_id': 'req-3263777c-7981-4449-b326-d7e007143e61'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:13.229090 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:13.242761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] The token cache is being invalidate because project f1faed53dd6541ecb1717e855e5d7f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:13.243532 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:13.244796 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3263777c-7981-4449-b326-d7e007143e61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:13.247818 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1432/2820] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:12 2022] DELETE /identity/v3/projects/f1faed53dd6541ecb1717e855e5d7f32 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:13.257421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06357371-7e88-4463-8ced-900658715641 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:13.259553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06357371-7e88-4463-8ced-900658715641 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:13.260068 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06357371-7e88-4463-8ced-900658715641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:13.297436 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-06357371-7e88-4463-8ced-900658715641 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:13.299933 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-06357371-7e88-4463-8ced-900658715641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:13.396982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1389/2821] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:13.406600 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355f3cf-b4da-4bd7-b679-2230dc699a90 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:13.407106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355f3cf-b4da-4bd7-b679-2230dc699a90 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:13.407713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355f3cf-b4da-4bd7-b679-2230dc699a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:13.438297 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6355f3cf-b4da-4bd7-b679-2230dc699a90 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:13.439581 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6355f3cf-b4da-4bd7-b679-2230dc699a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:13.452130 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1433/2822] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:13.453691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06357371-7e88-4463-8ced-900658715641 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:13.464501 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06357371-7e88-4463-8ced-900658715641 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:13.469676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3016c-3b1a-4cde-9fcd-25f0fccacd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:13.470405 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3016c-3b1a-4cde-9fcd-25f0fccacd27 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:13.470715 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3016c-3b1a-4cde-9fcd-25f0fccacd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:13.470947 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3016c-3b1a-4cde-9fcd-25f0fccacd27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:13.471718 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3016c-3b1a-4cde-9fcd-25f0fccacd27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:31:13.472019 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3016c-3b1a-4cde-9fcd-25f0fccacd27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:13.473795 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3016c-3b1a-4cde-9fcd-25f0fccacd27 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:13.496744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1390/2823] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:31:13.499194 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355f3cf-b4da-4bd7-b679-2230dc699a90 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:13.515086 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355f3cf-b4da-4bd7-b679-2230dc699a90 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:13.520933 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398279e-ab88-4ae1-bb4a-0eb0641ce759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:13.521663 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398279e-ab88-4ae1-bb4a-0eb0641ce759 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:13.522125 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398279e-ab88-4ae1-bb4a-0eb0641ce759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:13.522808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398279e-ab88-4ae1-bb4a-0eb0641ce759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:13.524021 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398279e-ab88-4ae1-bb4a-0eb0641ce759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:13.526204 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398279e-ab88-4ae1-bb4a-0eb0641ce759 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:13.576104 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1434/2824] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:13 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:13.577733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3016c-3b1a-4cde-9fcd-25f0fccacd27 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:13.597583 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3016c-3b1a-4cde-9fcd-25f0fccacd27 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:13.715988 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1853c-e297-4a81-aa82-0b11a5e00758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:13.717324 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1853c-e297-4a81-aa82-0b11a5e00758 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:13.717872 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1853c-e297-4a81-aa82-0b11a5e00758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:13.718381 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1853c-e297-4a81-aa82-0b11a5e00758 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:13.719793 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1853c-e297-4a81-aa82-0b11a5e00758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:13.723092 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1853c-e297-4a81-aa82-0b11a5e00758 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:13.790589 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1391/2825] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:13 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:13.790867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398279e-ab88-4ae1-bb4a-0eb0641ce759 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:13.806932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398279e-ab88-4ae1-bb4a-0eb0641ce759 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:13.815040 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775192a-0b55-47ad-a3f0-d23e00603a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:13.816523 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775192a-0b55-47ad-a3f0-d23e00603a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:13.816974 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775192a-0b55-47ad-a3f0-d23e00603a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:13.817534 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775192a-0b55-47ad-a3f0-d23e00603a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:13.821103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775192a-0b55-47ad-a3f0-d23e00603a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:13.824919 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775192a-0b55-47ad-a3f0-d23e00603a1d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:13.860716 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1435/2826] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:31:13.862222 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1853c-e297-4a81-aa82-0b11a5e00758 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:13.884000 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1853c-e297-4a81-aa82-0b11a5e00758 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:13.889254 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea28f0d-7f3d-4d18-a436-3ecf6509a8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:13.889977 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea28f0d-7f3d-4d18-a436-3ecf6509a8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:13.890271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea28f0d-7f3d-4d18-a436-3ecf6509a8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:13.890551 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea28f0d-7f3d-4d18-a436-3ecf6509a8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:13.891319 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea28f0d-7f3d-4d18-a436-3ecf6509a8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:13.892979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea28f0d-7f3d-4d18-a436-3ecf6509a8a1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:13.909750 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1392/2827] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:13.915123 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775192a-0b55-47ad-a3f0-d23e00603a1d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:13.933792 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775192a-0b55-47ad-a3f0-d23e00603a1d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:13.939450 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f263ca4-dda5-468b-ae66-709497daac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:13.940521 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f263ca4-dda5-468b-ae66-709497daac16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:13.940909 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f263ca4-dda5-468b-ae66-709497daac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:13.941261 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f263ca4-dda5-468b-ae66-709497daac16 admin admin] PATH_INFO: `/v3/projects/f9c5d938a69745c287d244262d6ab3f0/users/a83e83ef9acd4c498817e0e8b3e81d6c/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:13.963931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f263ca4-dda5-468b-ae66-709497daac16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c5d938a69745c287d244262d6ab3f0, user_id=a83e83ef9acd4c498817e0e8b3e81d6c, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:13.970932 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f263ca4-dda5-468b-ae66-709497daac16 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:14.003172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1436/2828] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:13 2022] PUT /identity/v3/projects/f9c5d938a69745c287d244262d6ab3f0/users/a83e83ef9acd4c498817e0e8b3e81d6c/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:14.018553 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c8481-2835-4609-bd7e-d95c830bd53c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:14.020308 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c8481-2835-4609-bd7e-d95c830bd53c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:14.021451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c8481-2835-4609-bd7e-d95c830bd53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:14.091229 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-400c8481-2835-4609-bd7e-d95c830bd53c None None] MFA Rules not processed for user `a83e83ef9acd4c498817e0e8b3e81d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:14.093668 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-400c8481-2835-4609-bd7e-d95c830bd53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:14.110965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:14.212846 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1393/2829] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:14.220245 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f263ca4-dda5-468b-ae66-709497daac16 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:14.401506 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f263ca4-dda5-468b-ae66-709497daac16 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:14.417301 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ca4a-637f-4164-bf4b-0ac2f0a79d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:14.421541 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ca4a-637f-4164-bf4b-0ac2f0a79d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:14.422084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ca4a-637f-4164-bf4b-0ac2f0a79d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:14.422652 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ca4a-637f-4164-bf4b-0ac2f0a79d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:14.424305 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355ca4a-637f-4164-bf4b-0ac2f0a79d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:14.430244 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355ca4a-637f-4164-bf4b-0ac2f0a79d2f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:14.498881 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1437/2830] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:14 2022] POST /identity/v3/projects => generated 381 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:14.501027 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c8481-2835-4609-bd7e-d95c830bd53c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:14.532491 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c8481-2835-4609-bd7e-d95c830bd53c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:14.657899 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465bd00-115e-4355-a526-df0bd6502a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:14.659313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465bd00-115e-4355-a526-df0bd6502a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:14.659608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465bd00-115e-4355-a526-df0bd6502a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:14.659884 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465bd00-115e-4355-a526-df0bd6502a4b admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:14.660636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465bd00-115e-4355-a526-df0bd6502a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:14.663451 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465bd00-115e-4355-a526-df0bd6502a4b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:14.698906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ca4a-637f-4164-bf4b-0ac2f0a79d2f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:14.701201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1394/2831] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:14 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:14.711680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ca4a-637f-4164-bf4b-0ac2f0a79d2f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:14.717799 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734f5e-e8a3-4df6-b885-1b63da98aa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:14.719099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734f5e-e8a3-4df6-b885-1b63da98aa3d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:14.719099 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734f5e-e8a3-4df6-b885-1b63da98aa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:14.719454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734f5e-e8a3-4df6-b885-1b63da98aa3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:14.720061 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9734f5e-e8a3-4df6-b885-1b63da98aa3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:14.721956 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9734f5e-e8a3-4df6-b885-1b63da98aa3d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:14.740972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1438/2832] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:31:14.746244 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465bd00-115e-4355-a526-df0bd6502a4b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:14.770313 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465bd00-115e-4355-a526-df0bd6502a4b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:14.780760 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9185315-ec25-4896-8b8f-52c75d195949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:14.782399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9185315-ec25-4896-8b8f-52c75d195949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:14.783170 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9185315-ec25-4896-8b8f-52c75d195949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:14.783682 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9185315-ec25-4896-8b8f-52c75d195949 admin admin] PATH_INFO: `/v3/projects/e6d27cbea0b4411cb28296f1037b8dad/users/fcef988376604e81bae81f6f3c92cfe1/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:14.815986 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9185315-ec25-4896-8b8f-52c75d195949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d27cbea0b4411cb28296f1037b8dad, user_id=fcef988376604e81bae81f6f3c92cfe1, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:14.823409 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9185315-ec25-4896-8b8f-52c75d195949 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:14.851162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1395/2833] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:14 2022] PUT /identity/v3/projects/e6d27cbea0b4411cb28296f1037b8dad/users/fcef988376604e81bae81f6f3c92cfe1/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:14.853797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734f5e-e8a3-4df6-b885-1b63da98aa3d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:14.874120 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734f5e-e8a3-4df6-b885-1b63da98aa3d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:15.004832 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33c45b-bb36-4197-b5d7-e47d1052b77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:15.006254 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33c45b-bb36-4197-b5d7-e47d1052b77a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:15.006999 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33c45b-bb36-4197-b5d7-e47d1052b77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:15.007595 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33c45b-bb36-4197-b5d7-e47d1052b77a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:15.009405 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33c45b-bb36-4197-b5d7-e47d1052b77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:15.013989 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33c45b-bb36-4197-b5d7-e47d1052b77a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:15.026825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:15.038740 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1439/2834] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:15.043707 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9185315-ec25-4896-8b8f-52c75d195949 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:15.062579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9185315-ec25-4896-8b8f-52c75d195949 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:15.081177 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20886c6d-65ba-4562-b92b-a0115cb014b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:15.083461 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20886c6d-65ba-4562-b92b-a0115cb014b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:15.084085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20886c6d-65ba-4562-b92b-a0115cb014b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:15.084697 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20886c6d-65ba-4562-b92b-a0115cb014b5 admin admin] PATH_INFO: `/v3/projects/e6d27cbea0b4411cb28296f1037b8dad/users/fcef988376604e81bae81f6f3c92cfe1/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:15.090675 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20886c6d-65ba-4562-b92b-a0115cb014b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d27cbea0b4411cb28296f1037b8dad, user_id=fcef988376604e81bae81f6f3c92cfe1, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:15.095162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20886c6d-65ba-4562-b92b-a0115cb014b5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:15.119531 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1396/2835] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:15 2022] PUT /identity/v3/projects/e6d27cbea0b4411cb28296f1037b8dad/users/fcef988376604e81bae81f6f3c92cfe1/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:15.141055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fc01b-1d61-4447-9c24-0b2ba3f6c0df None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:15.149466 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fc01b-1d61-4447-9c24-0b2ba3f6c0df None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:15.150119 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fc01b-1d61-4447-9c24-0b2ba3f6c0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:15.197276 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9c0fc01b-1d61-4447-9c24-0b2ba3f6c0df None None] MFA Rules not processed for user `fcef988376604e81bae81f6f3c92cfe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:15.199143 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9c0fc01b-1d61-4447-9c24-0b2ba3f6c0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:15.359316 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1440/2836] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:15.380762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20886c6d-65ba-4562-b92b-a0115cb014b5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:15.385468 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-20886c6d-65ba-4562-b92b-a0115cb014b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:15.514086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20886c6d-65ba-4562-b92b-a0115cb014b5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:15.521236 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ef1b0-148a-47c8-b29d-b4c3f4e51303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:15.522117 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ef1b0-148a-47c8-b29d-b4c3f4e51303 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:15.522716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ef1b0-148a-47c8-b29d-b4c3f4e51303 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:15.522821 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ef1b0-148a-47c8-b29d-b4c3f4e51303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:15.536524 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ef1b0-148a-47c8-b29d-b4c3f4e51303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:15.538819 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ef1b0-148a-47c8-b29d-b4c3f4e51303 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:15.561494 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1397/2837] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:15.714558 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fc01b-1d61-4447-9c24-0b2ba3f6c0df None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:15.761825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fc01b-1d61-4447-9c24-0b2ba3f6c0df None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:15.780565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bec09-2daf-4160-9323-d935f007ff23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '720c995318a24e7b859fa82febd8178b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:15.781738 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bec09-2daf-4160-9323-d935f007ff23 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:15.784445 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bec09-2daf-4160-9323-d935f007ff23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:15.785103 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bec09-2daf-4160-9323-d935f007ff23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:15.921149 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bec09-2daf-4160-9323-d935f007ff23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:15.924493 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bec09-2daf-4160-9323-d935f007ff23 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:15.947572 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1441/2838] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:16.568307 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ef1b0-148a-47c8-b29d-b4c3f4e51303 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:16.576346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:16.700672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ef1b0-148a-47c8-b29d-b4c3f4e51303 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:16.714799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:16.715885 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:16.716333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:16.717043 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] PATH_INFO: `/v3/users/a83e83ef9acd4c498817e0e8b3e81d6c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:16.718972 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83e83ef9acd4c498817e0e8b3e81d6c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:16.722128 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:16.816734 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83e83ef9acd4c498817e0e8b3e81d6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:16.829193 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83e83ef9acd4c498817e0e8b3e81d6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:16.830344 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83e83ef9acd4c498817e0e8b3e81d6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:16.859435 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83e83ef9acd4c498817e0e8b3e81d6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:16.860542 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83e83ef9acd4c498817e0e8b3e81d6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:16.898242 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1398/2839] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:16 2022] DELETE /identity/v3/users/a83e83ef9acd4c498817e0e8b3e81d6c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:16.905114 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b0e1e-54c3-4dbc-a4e2-b0679aace2a1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:16.905608 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b0e1e-54c3-4dbc-a4e2-b0679aace2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:16.905892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b0e1e-54c3-4dbc-a4e2-b0679aace2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:16.941527 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a59b0e1e-54c3-4dbc-a4e2-b0679aace2a1 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:16.945472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a59b0e1e-54c3-4dbc-a4e2-b0679aace2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:17.128546 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1442/2840] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:17.141777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:17.143404 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:17.207584 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab4f8f-3101-44e5-8cf1-293a8f09d63c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:17.208241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab4f8f-3101-44e5-8cf1-293a8f09d63c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:17.208821 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab4f8f-3101-44e5-8cf1-293a8f09d63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:17.244411 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da836b-036a-476f-9e19-76664a2cbf09 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:17.248786 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a5959-d22f-42d0-bfdd-9de20e38a970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:17.249806 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a5959-d22f-42d0-bfdd-9de20e38a970 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:17.250270 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a5959-d22f-42d0-bfdd-9de20e38a970 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:17.250740 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a5959-d22f-42d0-bfdd-9de20e38a970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:17.259236 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-03ab4f8f-3101-44e5-8cf1-293a8f09d63c None None] MFA Rules not processed for user `c795167b3e194db6925da938d9572912`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:17.260666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-03ab4f8f-3101-44e5-8cf1-293a8f09d63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:17.267088 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a5959-d22f-42d0-bfdd-9de20e38a970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:17.269304 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a5959-d22f-42d0-bfdd-9de20e38a970 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:17.287561 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1399/2841] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 14:31:17.293480 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:17.378116 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1443/2842] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:17.383374 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a5959-d22f-42d0-bfdd-9de20e38a970 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:17.385598 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-170a5959-d22f-42d0-bfdd-9de20e38a970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:17.400507 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a5959-d22f-42d0-bfdd-9de20e38a970 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:17.406104 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11a580-4f5a-4d6a-966c-50004d306937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:17.406830 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11a580-4f5a-4d6a-966c-50004d306937 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:17.407114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11a580-4f5a-4d6a-966c-50004d306937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:17.407406 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11a580-4f5a-4d6a-966c-50004d306937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:17.408174 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11a580-4f5a-4d6a-966c-50004d306937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:17.410208 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11a580-4f5a-4d6a-966c-50004d306937 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:17.443512 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1400/2843] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:17 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:17.448241 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab4f8f-3101-44e5-8cf1-293a8f09d63c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:17.471379 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab4f8f-3101-44e5-8cf1-293a8f09d63c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:17.589081 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa56483-3aea-45cb-b344-9389409d1729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:17.589936 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa56483-3aea-45cb-b344-9389409d1729 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:17.590440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa56483-3aea-45cb-b344-9389409d1729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:17.590736 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa56483-3aea-45cb-b344-9389409d1729 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:17.591696 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa56483-3aea-45cb-b344-9389409d1729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:17.593825 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa56483-3aea-45cb-b344-9389409d1729 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:17.624012 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1444/2844] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:17 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:17.626877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11a580-4f5a-4d6a-966c-50004d306937 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:17.639389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:17.645402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11a580-4f5a-4d6a-966c-50004d306937 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:17.655928 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd26e-d57f-4de9-8636-b33511d7c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:17.659139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd26e-d57f-4de9-8636-b33511d7c44e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:17.659770 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd26e-d57f-4de9-8636-b33511d7c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:17.660343 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd26e-d57f-4de9-8636-b33511d7c44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:17.661967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafd26e-d57f-4de9-8636-b33511d7c44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:17.665971 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafd26e-d57f-4de9-8636-b33511d7c44e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:17.685744 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1401/2845] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 14:31:17.692214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa56483-3aea-45cb-b344-9389409d1729 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:17.711725 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa56483-3aea-45cb-b344-9389409d1729 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:17.728421 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7108ed7-c921-403f-9783-0f5e769fa802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:17.728901 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd26e-d57f-4de9-8636-b33511d7c44e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:17.729916 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7108ed7-c921-403f-9783-0f5e769fa802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:17.730934 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7108ed7-c921-403f-9783-0f5e769fa802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:17.731441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7108ed7-c921-403f-9783-0f5e769fa802 admin admin] PATH_INFO: `/v3/projects/f7c5f6c25b1f4b9c8c36ce722075e10b/users/409d91f6fabe47b4ad2354fe7af657ce/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:17.731860 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ceafd26e-d57f-4de9-8636-b33511d7c44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:17.749924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd26e-d57f-4de9-8636-b33511d7c44e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:17.757481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:17.758979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:17.759265 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:17.759738 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] PATH_INFO: `/v3/projects/f9c5d938a69745c287d244262d6ab3f0` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:17.761396 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c5d938a69745c287d244262d6ab3f0)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:17.763769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:17.765322 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7108ed7-c921-403f-9783-0f5e769fa802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c5f6c25b1f4b9c8c36ce722075e10b, user_id=409d91f6fabe47b4ad2354fe7af657ce, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:17.771199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7108ed7-c921-403f-9783-0f5e769fa802 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:17.820748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1445/2846] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:17 2022] PUT /identity/v3/projects/f7c5f6c25b1f4b9c8c36ce722075e10b/users/409d91f6fabe47b4ad2354fe7af657ce/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:17.822510 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7108ed7-c921-403f-9783-0f5e769fa802 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:17.889299 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c5d938a69745c287d244262d6ab3f0', 'request_id': 'req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:17.901195 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c5d938a69745c287d244262d6ab3f0', 'request_id': 'req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:17.915502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c5d938a69745c287d244262d6ab3f0', 'request_id': 'req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:17.929213 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7108ed7-c921-403f-9783-0f5e769fa802 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:17.942662 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] The token cache is being invalidate because project f9c5d938a69745c287d244262d6ab3f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:17.943139 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:17.943924 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:17.947354 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1402/2847] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:17 2022] DELETE /identity/v3/projects/f9c5d938a69745c287d244262d6ab3f0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:17.953537 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:17.956194 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:17.958362 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:18.021913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2d7d4-28ac-4c57-8c55-f159a591cf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:18.022920 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2d7d4-28ac-4c57-8c55-f159a591cf71 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:18.023511 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2d7d4-28ac-4c57-8c55-f159a591cf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:18.023893 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2d7d4-28ac-4c57-8c55-f159a591cf71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:18.024737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2d7d4-28ac-4c57-8c55-f159a591cf71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:18.026609 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2d7d4-28ac-4c57-8c55-f159a591cf71 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:18.047438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1446/2848] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:31:18.049141 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2d7d4-28ac-4c57-8c55-f159a591cf71 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:18.051997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c6c2d7d4-28ac-4c57-8c55-f159a591cf71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:18.072832 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10aa9b-ceae-4470-98b5-0a7454aa2c37 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:18.074704 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2d7d4-28ac-4c57-8c55-f159a591cf71 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:18.078150 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:18.079050 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:18.079333 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:18.079617 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] PATH_INFO: `/v3/users/fcef988376604e81bae81f6f3c92cfe1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:18.080930 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcef988376604e81bae81f6f3c92cfe1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:18.082936 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:18.083803 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3fc3a-cba9-4a3e-991a-f6275539d28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:18.085554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3fc3a-cba9-4a3e-991a-f6275539d28c admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:18.086152 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3fc3a-cba9-4a3e-991a-f6275539d28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:18.086748 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3fc3a-cba9-4a3e-991a-f6275539d28c admin admin] PATH_INFO: `/v3/projects/f7c5f6c25b1f4b9c8c36ce722075e10b/users/409d91f6fabe47b4ad2354fe7af657ce/roles/d6ad1e18a93d4557a8581218e0802d90` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:18.089939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3fc3a-cba9-4a3e-991a-f6275539d28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c5f6c25b1f4b9c8c36ce722075e10b, user_id=409d91f6fabe47b4ad2354fe7af657ce, role_id=d6ad1e18a93d4557a8581218e0802d90)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:18.094331 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3fc3a-cba9-4a3e-991a-f6275539d28c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:18.122575 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1447/2849] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:18 2022] PUT /identity/v3/projects/f7c5f6c25b1f4b9c8c36ce722075e10b/users/409d91f6fabe47b4ad2354fe7af657ce/roles/d6ad1e18a93d4557a8581218e0802d90 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:18.172672 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcef988376604e81bae81f6f3c92cfe1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:18.185389 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcef988376604e81bae81f6f3c92cfe1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:18.186792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcef988376604e81bae81f6f3c92cfe1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:18.210602 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcef988376604e81bae81f6f3c92cfe1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:18.211762 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9bc9c0f5-bb3d-4dce-bbb2-37d3d656d28a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcef988376604e81bae81f6f3c92cfe1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:18.218328 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:18.237952 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1403/2850] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:17 2022] DELETE /identity/v3/users/fcef988376604e81bae81f6f3c92cfe1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:18.854544 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3fc3a-cba9-4a3e-991a-f6275539d28c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:18.858157 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d3d3fc3a-cba9-4a3e-991a-f6275539d28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:18.972548 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3fc3a-cba9-4a3e-991a-f6275539d28c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:19.057484 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:19.058761 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:19.059686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:19.060065 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] PATH_INFO: `/v3/projects/e6d27cbea0b4411cb28296f1037b8dad` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:19.061685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d27cbea0b4411cb28296f1037b8dad)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:19.063846 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:19.155127 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d27cbea0b4411cb28296f1037b8dad', 'request_id': 'req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:19.172561 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d27cbea0b4411cb28296f1037b8dad', 'request_id': 'req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:19.180567 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:19.199217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d27cbea0b4411cb28296f1037b8dad', 'request_id': 'req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:19.223754 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] The token cache is being invalidate because project e6d27cbea0b4411cb28296f1037b8dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:19.224795 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:19.226826 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d3c5fc42-9e17-4e99-a3ae-e3e9e793b482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:19.229481 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1448/2851] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:18 2022] DELETE /identity/v3/projects/e6d27cbea0b4411cb28296f1037b8dad => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:19.236236 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9f5c4-b955-4abd-8c69-5dada8e4c3e6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:19.237058 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9f5c4-b955-4abd-8c69-5dada8e4c3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:19.237618 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9f5c4-b955-4abd-8c69-5dada8e4c3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:19.269569 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d0e9f5c4-b955-4abd-8c69-5dada8e4c3e6 None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:19.271341 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d0e9f5c4-b955-4abd-8c69-5dada8e4c3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:19.380636 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1404/2852] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:19.383088 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2301ef-9947-42e3-b525-68c7ea06c59c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:19.383375 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2301ef-9947-42e3-b525-68c7ea06c59c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:19.383656 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2301ef-9947-42e3-b525-68c7ea06c59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:19.408037 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-aa2301ef-9947-42e3-b525-68c7ea06c59c None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:19.410603 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-aa2301ef-9947-42e3-b525-68c7ea06c59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:19.431391 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1449/2853] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:19.434144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9f5c4-b955-4abd-8c69-5dada8e4c3e6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:19.453449 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9f5c4-b955-4abd-8c69-5dada8e4c3e6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:19.465018 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481975e2-668e-4fad-add8-118b2df49079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:19.465863 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481975e2-668e-4fad-add8-118b2df49079 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:19.466419 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481975e2-668e-4fad-add8-118b2df49079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:19.466847 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481975e2-668e-4fad-add8-118b2df49079 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:19.467849 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481975e2-668e-4fad-add8-118b2df49079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 14:31:19.468331 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481975e2-668e-4fad-add8-118b2df49079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:19.470078 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481975e2-668e-4fad-add8-118b2df49079 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:19.486167 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1405/2854] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 6 14:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 14:31:19.491976 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2301ef-9947-42e3-b525-68c7ea06c59c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:19.512602 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2301ef-9947-42e3-b525-68c7ea06c59c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:19.530901 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766171d-7245-418d-ab65-938da3183793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:19.532231 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766171d-7245-418d-ab65-938da3183793 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:19.534224 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766171d-7245-418d-ab65-938da3183793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:19.534766 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766171d-7245-418d-ab65-938da3183793 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:19.536179 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766171d-7245-418d-ab65-938da3183793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:19.540133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766171d-7245-418d-ab65-938da3183793 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:19.592172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1450/2855] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 6 14:31:19 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 14:31:19.598482 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481975e2-668e-4fad-add8-118b2df49079 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:19.618732 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481975e2-668e-4fad-add8-118b2df49079 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:19.728733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61baf48f-12f4-4d2c-9f91-4e8c486dd6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:19.728733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61baf48f-12f4-4d2c-9f91-4e8c486dd6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:19.728733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61baf48f-12f4-4d2c-9f91-4e8c486dd6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:19.728733 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61baf48f-12f4-4d2c-9f91-4e8c486dd6a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:19.733379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61baf48f-12f4-4d2c-9f91-4e8c486dd6a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:19.733379 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61baf48f-12f4-4d2c-9f91-4e8c486dd6a6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:19.766300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1406/2856] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 6 14:31:19 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 14:31:19.772207 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766171d-7245-418d-ab65-938da3183793 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:19.788524 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766171d-7245-418d-ab65-938da3183793 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:19.809933 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1c9c4-21c1-41ba-aaa0-4d0c5f1c85d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:19.810637 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1c9c4-21c1-41ba-aaa0-4d0c5f1c85d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:19.810923 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1c9c4-21c1-41ba-aaa0-4d0c5f1c85d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:19.811198 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1c9c4-21c1-41ba-aaa0-4d0c5f1c85d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:19.811953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1c9c4-21c1-41ba-aaa0-4d0c5f1c85d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:19.813605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1c9c4-21c1-41ba-aaa0-4d0c5f1c85d2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:19.845113 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1451/2857] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:31:19.853728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61baf48f-12f4-4d2c-9f91-4e8c486dd6a6 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:19.880471 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61baf48f-12f4-4d2c-9f91-4e8c486dd6a6 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:19.890980 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787572b-e660-4c2a-94ea-d9815ddec27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:19.893118 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787572b-e660-4c2a-94ea-d9815ddec27b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:19.893904 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787572b-e660-4c2a-94ea-d9815ddec27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:19.894480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787572b-e660-4c2a-94ea-d9815ddec27b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:19.896165 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787572b-e660-4c2a-94ea-d9815ddec27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:19.900243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787572b-e660-4c2a-94ea-d9815ddec27b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:19.931474 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1407/2858] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 6 14:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 14:31:19.932877 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1c9c4-21c1-41ba-aaa0-4d0c5f1c85d2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:19.945318 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1c9c4-21c1-41ba-aaa0-4d0c5f1c85d2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:19.951223 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61c234-f2fc-4ae9-a84e-05d1bb938b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:19.952509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61c234-f2fc-4ae9-a84e-05d1bb938b65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:19.952964 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61c234-f2fc-4ae9-a84e-05d1bb938b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:19.953601 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61c234-f2fc-4ae9-a84e-05d1bb938b65 admin admin] PATH_INFO: `/v3/projects/91d223828a464158b40f1362fd4ffa8c/users/8d9e1d2f11614b179e938eb8f616e945/roles/4a06d59562de4d598671a462e3eaf62a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:19.972354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61c234-f2fc-4ae9-a84e-05d1bb938b65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d223828a464158b40f1362fd4ffa8c, user_id=8d9e1d2f11614b179e938eb8f616e945, role_id=4a06d59562de4d598671a462e3eaf62a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:19.976425 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61c234-f2fc-4ae9-a84e-05d1bb938b65 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:19.999904 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1452/2859] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 6 14:31:19 2022] PUT /identity/v3/projects/91d223828a464158b40f1362fd4ffa8c/users/8d9e1d2f11614b179e938eb8f616e945/roles/4a06d59562de4d598671a462e3eaf62a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 14:31:20.113994 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9c6b3-7dac-46f6-8b39-9e29fc866ccf None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:20.114587 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9c6b3-7dac-46f6-8b39-9e29fc866ccf None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:20.115882 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9c6b3-7dac-46f6-8b39-9e29fc866ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:20.167716 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2ad9c6b3-7dac-46f6-8b39-9e29fc866ccf None None] MFA Rules not processed for user `8d9e1d2f11614b179e938eb8f616e945`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:20.170237 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2ad9c6b3-7dac-46f6-8b39-9e29fc866ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:20.286475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:20.374765 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1408/2860] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:20.383398 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61c234-f2fc-4ae9-a84e-05d1bb938b65 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:20.385329 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-eb61c234-f2fc-4ae9-a84e-05d1bb938b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:20.565160 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61c234-f2fc-4ae9-a84e-05d1bb938b65 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:20.574346 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eaeda-7ac0-4485-8c20-705706a27812 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41a357957bc47739b7bd6f21d159d74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:20.575980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eaeda-7ac0-4485-8c20-705706a27812 service glance] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:20.576857 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eaeda-7ac0-4485-8c20-705706a27812 service glance] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:20.577642 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eaeda-7ac0-4485-8c20-705706a27812 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:20.600033 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76eaeda-7ac0-4485-8c20-705706a27812 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:20.605133 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76eaeda-7ac0-4485-8c20-705706a27812 service glance] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:20.634058 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1453/2861] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:24.767722 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfae16-aeb4-439e-9ce5-cd257a928d63 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:24.768301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfae16-aeb4-439e-9ce5-cd257a928d63 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:24.768560 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfae16-aeb4-439e-9ce5-cd257a928d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:24.807648 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-90dfae16-aeb4-439e-9ce5-cd257a928d63 None None] MFA Rules not processed for user `409d91f6fabe47b4ad2354fe7af657ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:24.808995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-90dfae16-aeb4-439e-9ce5-cd257a928d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:24.962179 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1409/2862] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 14:31:24.980455 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eaeda-7ac0-4485-8c20-705706a27812 service glance] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:24.982604 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d76eaeda-7ac0-4485-8c20-705706a27812 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:25.076440 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eaeda-7ac0-4485-8c20-705706a27812 service glance] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:25.082953 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afab60d-cd00-49dc-835f-668e3e75b676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:25.083632 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afab60d-cd00-49dc-835f-668e3e75b676 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:25.083908 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afab60d-cd00-49dc-835f-668e3e75b676 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:25.084184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afab60d-cd00-49dc-835f-668e3e75b676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:25.086718 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5afab60d-cd00-49dc-835f-668e3e75b676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:25.111937 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:25.162497 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afab60d-cd00-49dc-835f-668e3e75b676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:25.168169 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afab60d-cd00-49dc-835f-668e3e75b676 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:25.180435 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1454/2863] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:34.957807 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=38 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:35.174451 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=17 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:31:47.774905 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e0285-0fc9-4563-9332-0ca9578572ef None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:47.775442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e0285-0fc9-4563-9332-0ca9578572ef None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:47.775442 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e0285-0fc9-4563-9332-0ca9578572ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:47.792938 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-855e0285-0fc9-4563-9332-0ca9578572ef None None] MFA Rules not processed for user `c795167b3e194db6925da938d9572912`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:47.794144 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-855e0285-0fc9-4563-9332-0ca9578572ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:47.800877 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1410/2864] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:50.402138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afab60d-cd00-49dc-835f-668e3e75b676 service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:50.475866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afab60d-cd00-49dc-835f-668e3e75b676 service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:50.480441 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:50.481564 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:50.481894 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:50.482167 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] PATH_INFO: `/v3/users/8d9e1d2f11614b179e938eb8f616e945` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:50.483358 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9e1d2f11614b179e938eb8f616e945)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:50.485296 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:50.540772 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9e1d2f11614b179e938eb8f616e945'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:50.549013 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9e1d2f11614b179e938eb8f616e945'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:50.549874 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9e1d2f11614b179e938eb8f616e945'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:50.558554 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9e1d2f11614b179e938eb8f616e945'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:50.572418 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9e1d2f11614b179e938eb8f616e945'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:50.578638 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1455/2865] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:31:50 2022] DELETE /identity/v3/users/8d9e1d2f11614b179e938eb8f616e945 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:31:50.583228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda4ce7-d021-4359-a970-e9cf67d03daa None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:50.583739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda4ce7-d021-4359-a970-e9cf67d03daa None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:50.583915 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda4ce7-d021-4359-a970-e9cf67d03daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:50.602570 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5dda4ce7-d021-4359-a970-e9cf67d03daa None None] MFA Rules not processed for user `e2d188b242aa4be3ac73c99d08c98dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:31:50.603852 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5dda4ce7-d021-4359-a970-e9cf67d03daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:50.665209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1411/2866] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:31:50.680350 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:50.681808 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:50.747811 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb184e-a53b-423b-8fd6-17b6e5c988c3 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:50.751641 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4a598-0955-4294-9b8f-8ae56f403def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:50.752293 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4a598-0955-4294-9b8f-8ae56f403def service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:50.752578 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4a598-0955-4294-9b8f-8ae56f403def service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:50.752853 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4a598-0955-4294-9b8f-8ae56f403def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:50.763764 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4a598-0955-4294-9b8f-8ae56f403def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:50.765274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4a598-0955-4294-9b8f-8ae56f403def service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:50.776457 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1456/2867] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:31:51.246186 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda4ce7-d021-4359-a970-e9cf67d03daa None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:31:51.249028 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5dda4ce7-d021-4359-a970-e9cf67d03daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:31:51.263300 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda4ce7-d021-4359-a970-e9cf67d03daa None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:31:51.277189 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:31:51.278243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:31:51.279271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:31:51.279390 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] PATH_INFO: `/v3/projects/91d223828a464158b40f1362fd4ffa8c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:31:51.280873 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d223828a464158b40f1362fd4ffa8c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:31:51.283481 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:31:51.356608 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d223828a464158b40f1362fd4ffa8c', 'request_id': 'req-bee95ea5-547c-42cb-ae86-5097ca626c59'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:51.368185 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d223828a464158b40f1362fd4ffa8c', 'request_id': 'req-bee95ea5-547c-42cb-ae86-5097ca626c59'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:51.376883 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d223828a464158b40f1362fd4ffa8c', 'request_id': 'req-bee95ea5-547c-42cb-ae86-5097ca626c59'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:51.390226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] The token cache is being invalidate because project 91d223828a464158b40f1362fd4ffa8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:31:51.390563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:51.391267 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:31:51.406047 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1412/2868] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:31:51 2022] DELETE /identity/v3/projects/91d223828a464158b40f1362fd4ffa8c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:32:00.772867 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:32:01.386737 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:32:02.892827 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4a598-0955-4294-9b8f-8ae56f403def service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:32:02.902789 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4a598-0955-4294-9b8f-8ae56f403def service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:32:02.951443 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c648fe-d822-4922-a928-373057bc7b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:32:02.952085 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c648fe-d822-4922-a928-373057bc7b36 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:32:02.952369 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c648fe-d822-4922-a928-373057bc7b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:32:02.952645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c648fe-d822-4922-a928-373057bc7b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:32:02.954438 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e0c648fe-d822-4922-a928-373057bc7b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:32:03.019691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c648fe-d822-4922-a928-373057bc7b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:32:03.021148 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c648fe-d822-4922-a928-373057bc7b36 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:32:03.031199 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1457/2869] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:32:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:32:13.026676 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=50 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:32:54.866085 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:32:54.867549 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:32:54.881467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee95ea5-547c-42cb-ae86-5097ca626c59 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:32:54.885287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:32:54.885979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:32:54.886298 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:32:54.886578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:32:54.888456 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:32:54.976115 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:32:54.978784 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:32:54.995979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1413/2870] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:32:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:32:56.175823 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c648fe-d822-4922-a928-373057bc7b36 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:32:56.184659 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c648fe-d822-4922-a928-373057bc7b36 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:32:56.188650 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ed7fd-b813-4894-b82f-80f271a7ebab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:32:56.189289 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ed7fd-b813-4894-b82f-80f271a7ebab service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:32:56.189571 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ed7fd-b813-4894-b82f-80f271a7ebab service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:32:56.189892 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ed7fd-b813-4894-b82f-80f271a7ebab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:32:56.191661 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d06ed7fd-b813-4894-b82f-80f271a7ebab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:32:56.205345 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ed7fd-b813-4894-b82f-80f271a7ebab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:32:56.206810 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ed7fd-b813-4894-b82f-80f271a7ebab service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:32:56.218420 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1458/2871] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:32:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:33:04.987743 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=34 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:33:05.084777 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:33:05.086295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:33:05.098087 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bd26d-4785-4c91-9de7-ecc6cf80a564 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:33:05.101790 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703d9c-9db6-4dd2-a202-b30c2ac7df4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:33:05.102443 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703d9c-9db6-4dd2-a202-b30c2ac7df4d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:33:05.102726 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703d9c-9db6-4dd2-a202-b30c2ac7df4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:33:05.103008 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703d9c-9db6-4dd2-a202-b30c2ac7df4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:33:05.104815 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fb703d9c-9db6-4dd2-a202-b30c2ac7df4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:33:05.116457 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb703d9c-9db6-4dd2-a202-b30c2ac7df4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:33:05.117965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb703d9c-9db6-4dd2-a202-b30c2ac7df4d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:33:05.126864 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1414/2872] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:33:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:33:06.212794 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:33:09.426968 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bea44-d192-4bae-8b3c-4a233b1ac3d7 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:33:09.427674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bea44-d192-4bae-8b3c-4a233b1ac3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:33:09.427674 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bea44-d192-4bae-8b3c-4a233b1ac3d7 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:33:09.429172 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1459/2873] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:33:09.436865 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9badc-d492-4bca-938c-9d9aff3042c2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:33:09.437423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9badc-d492-4bca-938c-9d9aff3042c2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:33:09.437423 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9badc-d492-4bca-938c-9d9aff3042c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:33:09.476947 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2dd9badc-d492-4bca-938c-9d9aff3042c2 None None] MFA Rules not processed for user `59cd9d922204496d8ea185716b832481`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:33:09.478271 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2dd9badc-d492-4bca-938c-9d9aff3042c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:33:09.486135 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1415/2874] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:33:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:33:09.499053 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bea44-d192-4bae-8b3c-4a233b1ac3d7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:33:09.511997 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bea44-d192-4bae-8b3c-4a233b1ac3d7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:33:09.516184 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92984079-152c-4dcf-b7e0-652f7bbc1ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:33:09.516870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92984079-152c-4dcf-b7e0-652f7bbc1ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:33:09.517155 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92984079-152c-4dcf-b7e0-652f7bbc1ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:33:09.517432 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92984079-152c-4dcf-b7e0-652f7bbc1ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:33:09.525203 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92984079-152c-4dcf-b7e0-652f7bbc1ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:33:09.526870 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92984079-152c-4dcf-b7e0-652f7bbc1ac5 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:33:09.537691 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1460/2875] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:33:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:33:09.625065 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf88f-9d1e-4276-8e20-c4a7d6a63e23 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:33:09.625363 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf88f-9d1e-4276-8e20-c4a7d6a63e23 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:33:09.625647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf88f-9d1e-4276-8e20-c4a7d6a63e23 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:33:09.627083 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1416/2876] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 6 14:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 14:33:09.632685 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aceca-db9b-4388-837b-9fdafdc0f94c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:33:09.632980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aceca-db9b-4388-837b-9fdafdc0f94c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:33:09.633262 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aceca-db9b-4388-837b-9fdafdc0f94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:33:09.651797 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-da9aceca-db9b-4388-837b-9fdafdc0f94c None None] MFA Rules not processed for user `0b82a920df4042dd97314350aed4d7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:33:09.652957 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-da9aceca-db9b-4388-837b-9fdafdc0f94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:33:09.663274 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1461/2877] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 6 14:33:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:33:09.674160 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbf88f-9d1e-4276-8e20-c4a7d6a63e23 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:33:09.685362 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbf88f-9d1e-4276-8e20-c4a7d6a63e23 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:33:09.688990 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f71479-dcf3-4fdc-b765-f0c698381a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:33:09.689627 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f71479-dcf3-4fdc-b765-f0c698381a7a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:33:09.689949 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f71479-dcf3-4fdc-b765-f0c698381a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:33:09.690230 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f71479-dcf3-4fdc-b765-f0c698381a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:33:09.700515 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f71479-dcf3-4fdc-b765-f0c698381a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:33:09.701969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f71479-dcf3-4fdc-b765-f0c698381a7a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:33:09.712906 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1417/2878] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:33:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:33:09.774452 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aceca-db9b-4388-837b-9fdafdc0f94c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:33:09.775824 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-da9aceca-db9b-4388-837b-9fdafdc0f94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:33:09.849292 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aceca-db9b-4388-837b-9fdafdc0f94c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:33:09.854282 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:33:09.854931 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:33:09.855214 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:33:09.855489 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:33:09.858317 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:33:09.870543 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:33:09.872501 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:33:09.883612 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1462/2879] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:33:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:33:19.708134 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=21 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:33:19.879084 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=41 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:33:21.366625 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f71479-dcf3-4fdc-b765-f0c698381a7a service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:33:21.368086 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d5f71479-dcf3-4fdc-b765-f0c698381a7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:33:21.384655 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f71479-dcf3-4fdc-b765-f0c698381a7a service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:33:21.392350 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcedaa-17e7-4920-8989-92787177b536 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:33:21.393023 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcedaa-17e7-4920-8989-92787177b536 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:33:21.393301 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcedaa-17e7-4920-8989-92787177b536 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:33:21.393578 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcedaa-17e7-4920-8989-92787177b536 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:33:21.395445 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-81bcedaa-17e7-4920-8989-92787177b536 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:33:21.407995 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcedaa-17e7-4920-8989-92787177b536 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:33:21.409572 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcedaa-17e7-4920-8989-92787177b536 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:33:21.421546 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1418/2880] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:33:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:33:31.416238 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:34:05.279630 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:34:05.281334 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:05.300159 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d986-46fd-460e-9b75-9abcc165c938 service placement] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:34:05.313283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbba9a-3e93-48c8-a45e-5243a7ab3b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:34:05.314030 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbba9a-3e93-48c8-a45e-5243a7ab3b9a service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:05.314321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbba9a-3e93-48c8-a45e-5243a7ab3b9a service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:05.314735 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbba9a-3e93-48c8-a45e-5243a7ab3b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:05.316831 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-86dbba9a-3e93-48c8-a45e-5243a7ab3b9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:05.330913 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbba9a-3e93-48c8-a45e-5243a7ab3b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:34:05.332533 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbba9a-3e93-48c8-a45e-5243a7ab3b9a service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:34:05.347891 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1463/2881] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:34:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:34:06.150228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b22e7c-97bf-407d-9a54-51e92f0f717b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:06.150746 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b22e7c-97bf-407d-9a54-51e92f0f717b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:06.150867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b22e7c-97bf-407d-9a54-51e92f0f717b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:06.175792 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-04b22e7c-97bf-407d-9a54-51e92f0f717b None None] MFA Rules not processed for user `c795167b3e194db6925da938d9572912`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:34:06.177002 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-04b22e7c-97bf-407d-9a54-51e92f0f717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:06.239665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1419/2882] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:34:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:34:06.565027 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbba9a-3e93-48c8-a45e-5243a7ab3b9a service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:34:06.574434 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbba9a-3e93-48c8-a45e-5243a7ab3b9a service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:34:06.579087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60741c-a8d3-487d-9ee9-759c16aba1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:34:06.579722 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60741c-a8d3-487d-9ee9-759c16aba1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:06.580006 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60741c-a8d3-487d-9ee9-759c16aba1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:06.580283 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60741c-a8d3-487d-9ee9-759c16aba1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:06.582075 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2d60741c-a8d3-487d-9ee9-759c16aba1c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:06.591756 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60741c-a8d3-487d-9ee9-759c16aba1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:34:06.593549 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60741c-a8d3-487d-9ee9-759c16aba1c1 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:34:06.602211 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1464/2883] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:34:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:34:16.231342 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=25 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:34:16.598171 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=12 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:34:40.683822 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b22e7c-97bf-407d-9a54-51e92f0f717b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:34:40.744603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b22e7c-97bf-407d-9a54-51e92f0f717b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:34:40.751703 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beccd5e-e3d8-4bb0-b33f-6b097645dead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:34:40.752475 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beccd5e-e3d8-4bb0-b33f-6b097645dead service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:40.752895 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beccd5e-e3d8-4bb0-b33f-6b097645dead service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:40.753295 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beccd5e-e3d8-4bb0-b33f-6b097645dead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:40.755874 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7beccd5e-e3d8-4bb0-b33f-6b097645dead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:40.823942 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beccd5e-e3d8-4bb0-b33f-6b097645dead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:34:40.827162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beccd5e-e3d8-4bb0-b33f-6b097645dead service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:34:40.852996 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1420/2884] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:34:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:34:44.860509 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60741c-a8d3-487d-9ee9-759c16aba1c1 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:34:44.912710 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60741c-a8d3-487d-9ee9-759c16aba1c1 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:34:44.919424 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:34:44.920101 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:44.920384 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:44.920660 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:44.923328 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:45.034178 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:34:45.035666 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:34:45.057587 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1465/2885] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:34:48.431817 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beccd5e-e3d8-4bb0-b33f-6b097645dead service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:34:48.445799 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beccd5e-e3d8-4bb0-b33f-6b097645dead service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:34:48.451317 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeba0e9-2895-4f0a-8e86-e8dca6d12dc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:34:48.452314 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeba0e9-2895-4f0a-8e86-e8dca6d12dc9 service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:48.453157 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeba0e9-2895-4f0a-8e86-e8dca6d12dc9 service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:48.453660 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeba0e9-2895-4f0a-8e86-e8dca6d12dc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:48.455564 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0eeba0e9-2895-4f0a-8e86-e8dca6d12dc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:48.477143 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeba0e9-2895-4f0a-8e86-e8dca6d12dc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:34:48.478979 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeba0e9-2895-4f0a-8e86-e8dca6d12dc9 service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:34:48.496484 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1421/2886] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:34:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:34:51.490946 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:34:51.492713 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:51.506983 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60179d7-9064-4ed8-bc97-e8157d86cf62 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:34:51.513008 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:34:51.513939 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:51.514225 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:51.514500 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] PATH_INFO: `/v3/users/4e867646e7734a24a574fcbce0b17c8b` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:51.516415 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e867646e7734a24a574fcbce0b17c8b)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:34:51.518433 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:34:51.574140 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e867646e7734a24a574fcbce0b17c8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.583219 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e867646e7734a24a574fcbce0b17c8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.583943 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e867646e7734a24a574fcbce0b17c8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.593454 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e867646e7734a24a574fcbce0b17c8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.606399 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e867646e7734a24a574fcbce0b17c8b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.613052 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1466/2887] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:34:51 2022] DELETE /identity/v3/users/4e867646e7734a24a574fcbce0b17c8b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:34:51.787091 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeba0e9-2895-4f0a-8e86-e8dca6d12dc9 service placement] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:34:51.803292 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeba0e9-2895-4f0a-8e86-e8dca6d12dc9 service placement] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:34:51.861502 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:34:51.862402 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:51.862690 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:51.862967 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] PATH_INFO: `/v3/projects/add30fb78c80414599ebb8f57fb31389` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:51.864200 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=add30fb78c80414599ebb8f57fb31389)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:34:51.866197 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:34:51.914348 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add30fb78c80414599ebb8f57fb31389', 'request_id': 'req-2b201853-96c0-4bdf-bdcc-26a138f25b96'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.922563 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add30fb78c80414599ebb8f57fb31389', 'request_id': 'req-2b201853-96c0-4bdf-bdcc-26a138f25b96'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.928291 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add30fb78c80414599ebb8f57fb31389', 'request_id': 'req-2b201853-96c0-4bdf-bdcc-26a138f25b96'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.940943 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] The token cache is being invalidate because project add30fb78c80414599ebb8f57fb31389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:34:51.941231 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.941903 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:51.943480 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1422/2888] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:34:51 2022] DELETE /identity/v3/projects/add30fb78c80414599ebb8f57fb31389 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:34:51.947619 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:34:51.949051 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:51.964834 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059117b8-bd4d-456a-89d4-c7b48a3d30cf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:34:51.974270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:34:52.026848 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:34:52.027732 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:52.028017 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:52.028291 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] PATH_INFO: `/v3/users/6c237ed331704ae2a43fdecfe860c13e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:52.029482 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c237ed331704ae2a43fdecfe860c13e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:34:52.032003 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:34:52.080866 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c237ed331704ae2a43fdecfe860c13e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.087863 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c237ed331704ae2a43fdecfe860c13e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.088513 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c237ed331704ae2a43fdecfe860c13e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.095865 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c237ed331704ae2a43fdecfe860c13e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.108253 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-de598101-3c0d-4df4-9aca-4bfeeb9c9b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c237ed331704ae2a43fdecfe860c13e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.114972 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1467/2889] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:34:51 2022] DELETE /identity/v3/users/6c237ed331704ae2a43fdecfe860c13e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:34:52.290778 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:34:52.292338 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:34:52.306472 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b201853-96c0-4bdf-bdcc-26a138f25b96 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:34:52.318761 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:34:52.367728 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:34:52.368588 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:34:52.368867 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:34:52.369138 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] PATH_INFO: `/v3/projects/046d374b343c49deaa23fb1fbdb85cbe` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:34:52.370332 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=046d374b343c49deaa23fb1fbdb85cbe)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:34:52.372209 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:34:52.419095 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046d374b343c49deaa23fb1fbdb85cbe', 'request_id': 'req-63e8e763-d045-4e8a-b1b4-79c090f01328'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.428828 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046d374b343c49deaa23fb1fbdb85cbe', 'request_id': 'req-63e8e763-d045-4e8a-b1b4-79c090f01328'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.435352 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046d374b343c49deaa23fb1fbdb85cbe', 'request_id': 'req-63e8e763-d045-4e8a-b1b4-79c090f01328'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.447376 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] The token cache is being invalidate because project 046d374b343c49deaa23fb1fbdb85cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:34:52.447739 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.448323 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:34:52.449907 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1423/2890] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:34:52 2022] DELETE /identity/v3/projects/046d374b343c49deaa23fb1fbdb85cbe => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:35:02.111536 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:35:02.444192 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:36:14.506565 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b052fc-f92b-40b5-9472-020dcc8876f0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:36:14.507138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b052fc-f92b-40b5-9472-020dcc8876f0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:36:14.507138 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b052fc-f92b-40b5-9472-020dcc8876f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:36:14.524686 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c8b052fc-f92b-40b5-9472-020dcc8876f0 None None] MFA Rules not processed for user `c795167b3e194db6925da938d9572912`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 14:36:14.525947 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c8b052fc-f92b-40b5-9472-020dcc8876f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:36:14.583046 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1468/2891] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 6 14:36:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 14:36:15.802401 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:36:15.803775 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:36:15.899114 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8e763-d045-4e8a-b1b4-79c090f01328 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:36:15.904279 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:36:15.904929 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:36:15.905213 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:36:15.905489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:36:15.907418 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:36:15.919676 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:36:15.921245 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:36:15.931679 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1424/2892] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:36:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:36:24.574906 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=25 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:36:25.743197 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b052fc-f92b-40b5-9472-020dcc8876f0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:36:25.744428 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c8b052fc-f92b-40b5-9472-020dcc8876f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:36:25.852677 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b052fc-f92b-40b5-9472-020dcc8876f0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:36:25.858783 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b82a920df4042dd97314350aed4d7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:36:25.859437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:36:25.859720 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:36:25.860117 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:36:25.863370 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:36:25.874270 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:36:25.875871 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:36:25.885862 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1469/2893] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:36:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:36:25.927303 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=41 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:36:35.881746 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=46 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:38:05.258504 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:38:05.259987 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:05.333584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a9713-27d0-4416-a758-b7f9e6d74b5a service placement] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:38:05.338012 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9045ab-78a0-4fd2-939c-0ac486d1f81d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:38:05.338665 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9045ab-78a0-4fd2-939c-0ac486d1f81d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:38:05.338965 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9045ab-78a0-4fd2-939c-0ac486d1f81d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:38:05.339228 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9045ab-78a0-4fd2-939c-0ac486d1f81d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:38:05.341037 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1a9045ab-78a0-4fd2-939c-0ac486d1f81d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:05.354859 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9045ab-78a0-4fd2-939c-0ac486d1f81d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:38:05.356287 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9045ab-78a0-4fd2-939c-0ac486d1f81d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:38:05.368305 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1425/2894] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:38:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:38:06.479174 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:38:06.480708 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:06.493849 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82159212-946d-4fe3-895f-f1b33137fb9a service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:38:06.498760 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8261db-b542-42f9-aa9e-133ad6f14101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:38:06.499402 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8261db-b542-42f9-aa9e-133ad6f14101 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:38:06.499682 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8261db-b542-42f9-aa9e-133ad6f14101 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:38:06.499959 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8261db-b542-42f9-aa9e-133ad6f14101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:38:06.501786 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4a8261db-b542-42f9-aa9e-133ad6f14101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:06.514313 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8261db-b542-42f9-aa9e-133ad6f14101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:38:06.515773 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8261db-b542-42f9-aa9e-133ad6f14101 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:38:06.526574 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1470/2895] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:38:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:38:06.552829 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9045ab-78a0-4fd2-939c-0ac486d1f81d service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:38:06.563969 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9045ab-78a0-4fd2-939c-0ac486d1f81d service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:38:06.567702 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b782d-2b3c-475d-9a50-47c9995242aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:38:06.568388 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b782d-2b3c-475d-9a50-47c9995242aa service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:38:06.568670 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b782d-2b3c-475d-9a50-47c9995242aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:38:06.569467 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b782d-2b3c-475d-9a50-47c9995242aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:38:06.571326 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-aa5b782d-2b3c-475d-9a50-47c9995242aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:06.585691 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b782d-2b3c-475d-9a50-47c9995242aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:38:06.587162 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b782d-2b3c-475d-9a50-47c9995242aa service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:38:06.598201 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1426/2896] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:38:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:38:16.520843 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:38:16.593255 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=37 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:38:17.391784 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8261db-b542-42f9-aa9e-133ad6f14101 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:38:17.401437 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8261db-b542-42f9-aa9e-133ad6f14101 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:38:17.405048 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59cd9d922204496d8ea185716b832481', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:38:17.405680 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:38:17.405998 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:38:17.406275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:38:17.408087 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:17.471629 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:38:17.473093 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:38:17.483165 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1471/2897] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:38:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:38:20.334489 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b782d-2b3c-475d-9a50-47c9995242aa service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:38:20.347574 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b782d-2b3c-475d-9a50-47c9995242aa service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:38:20.354399 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508f43f-994b-448e-bfef-48d738827cab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:38:20.355831 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508f43f-994b-448e-bfef-48d738827cab service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:38:20.356346 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508f43f-994b-448e-bfef-48d738827cab service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:38:20.356802 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508f43f-994b-448e-bfef-48d738827cab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:38:20.359004 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f508f43f-994b-448e-bfef-48d738827cab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:20.374080 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508f43f-994b-448e-bfef-48d738827cab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:38:20.376579 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508f43f-994b-448e-bfef-48d738827cab service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:38:20.393603 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1427/2898] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:38:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:38:23.586605 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:38:23.588106 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:23.600473 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d6215-7d13-444c-97b6-040092f04602 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:38:23.606309 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:38:23.607217 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:38:23.607502 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:38:23.607780 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] PATH_INFO: `/v3/users/c795167b3e194db6925da938d9572912` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:38:23.609216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c795167b3e194db6925da938d9572912)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:38:23.611176 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:38:23.657585 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c795167b3e194db6925da938d9572912'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:23.665321 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c795167b3e194db6925da938d9572912'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:23.666005 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c795167b3e194db6925da938d9572912'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:23.674744 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c795167b3e194db6925da938d9572912'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:23.687753 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c795167b3e194db6925da938d9572912'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:23.693878 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1472/2899] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:38:23 2022] DELETE /identity/v3/users/c795167b3e194db6925da938d9572912 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:38:23.874773 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508f43f-994b-448e-bfef-48d738827cab service placement] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:38:23.889647 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508f43f-994b-448e-bfef-48d738827cab service placement] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:38:23.948833 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:38:23.950365 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:38:23.950803 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:38:23.951266 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] PATH_INFO: `/v3/projects/8963278084b2491799c6c5e403b5cfcc` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:38:23.952927 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8963278084b2491799c6c5e403b5cfcc)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:38:23.954868 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:38:24.002839 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8963278084b2491799c6c5e403b5cfcc', 'request_id': 'req-12dd269f-ad3c-493c-88e6-d3ff79780d2a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.010687 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8963278084b2491799c6c5e403b5cfcc', 'request_id': 'req-12dd269f-ad3c-493c-88e6-d3ff79780d2a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.016421 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8963278084b2491799c6c5e403b5cfcc', 'request_id': 'req-12dd269f-ad3c-493c-88e6-d3ff79780d2a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.029947 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] The token cache is being invalidate because project 8963278084b2491799c6c5e403b5cfcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:38:24.030554 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.031635 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.033769 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1428/2900] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:38:23 2022] DELETE /identity/v3/projects/8963278084b2491799c6c5e403b5cfcc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:38:24.038098 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:38:24.039556 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:24.052645 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff1e1-c2c2-42c3-a330-579e5e7679e4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:38:24.111589 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:38:24.112472 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:38:24.112775 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:38:24.113055 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] PATH_INFO: `/v3/users/409d91f6fabe47b4ad2354fe7af657ce` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:38:24.114275 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=409d91f6fabe47b4ad2354fe7af657ce)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:38:24.116216 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:38:24.122471 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:38:24.168739 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409d91f6fabe47b4ad2354fe7af657ce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.175519 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409d91f6fabe47b4ad2354fe7af657ce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.176189 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409d91f6fabe47b4ad2354fe7af657ce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.184038 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409d91f6fabe47b4ad2354fe7af657ce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.197077 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409d91f6fabe47b4ad2354fe7af657ce'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.203062 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1473/2901] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 6 14:38:24 2022] DELETE /identity/v3/users/409d91f6fabe47b4ad2354fe7af657ce => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:38:24.381673 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:38:24.384164 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:38:24.403054 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd269f-ad3c-493c-88e6-d3ff79780d2a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:38:24.465334 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2d188b242aa4be3ac73c99d08c98dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '258fdfe108b84b6c82c61b2a2990c11d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:38:24.466226 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:38:24.466584 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:38:24.466861 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] PATH_INFO: `/v3/projects/f7c5f6c25b1f4b9c8c36ce722075e10b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:38:24.468066 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c5f6c25b1f4b9c8c36ce722075e10b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:38:24.469981 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:38:24.494049 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:38:24.522071 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c5f6c25b1f4b9c8c36ce722075e10b', 'request_id': 'req-0d170281-f2c8-463b-a2e9-a9286165e73b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.529654 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c5f6c25b1f4b9c8c36ce722075e10b', 'request_id': 'req-0d170281-f2c8-463b-a2e9-a9286165e73b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.535243 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c5f6c25b1f4b9c8c36ce722075e10b', 'request_id': 'req-0d170281-f2c8-463b-a2e9-a9286165e73b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.548667 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] The token cache is being invalidate because project f7c5f6c25b1f4b9c8c36ce722075e10b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 14:38:24.549051 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.549804 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d170281-f2c8-463b-a2e9-a9286165e73b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 14:38:24.556982 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1429/2902] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 6 14:38:24 2022] DELETE /identity/v3/projects/f7c5f6c25b1f4b9c8c36ce722075e10b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 14:38:34.199935 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:38:34.544925 dv-cinder-844140-2-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 14:41:20.960933 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 14:41:20.962367 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:41:21.031635 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49713-2705-4290-ac3c-b134c9c7db77 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 14:41:21.095518 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d26692-1b1e-4d8a-a05b-d986b6e6b4f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51b91456a0634e0fa22fb389f0721a65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2adb074ba2944fb82d24492b94b012b', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 14:41:21.096173 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d26692-1b1e-4d8a-a05b-d986b6e6b4f3 service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 14:41:21.096458 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d26692-1b1e-4d8a-a05b-d986b6e6b4f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 14:41:21.096737 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d26692-1b1e-4d8a-a05b-d986b6e6b4f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 14:41:21.098591 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a5d26692-1b1e-4d8a-a05b-d986b6e6b4f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 14:41:21.111354 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d26692-1b1e-4d8a-a05b-d986b6e6b4f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 14:41:21.112980 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d26692-1b1e-4d8a-a05b-d986b6e6b4f3 service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 14:41:21.123654 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1474/2903] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 6 14:41:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 14:41:31.119158 dv-cinder-844140-2-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=58 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}